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(57) Abstract 

This system locates and displays the lelative positions of one moveable object (12) with respect to a second moveable object (1 1) in 
a three-dimensional space. The locations of at least two points reference (14. 16) on the first object (12) and at least three reference points 
(73-75) on the second object (11) are measurable by position sensors (20. 22. 24) in a predetermined fixed coordinate systcna (80). From 
the locations of the reference points (14, 16, 73-75), a ccmputer (36) detennines the positions and orientations of the two objects (11. 12) 
in the fixed coordinate system (SO). From the positions and orientations of die two objects (11, 12), the computer (36) further determines 
the leladve position and orientation of die first object (12) with respect to a predetemiined local coordinate system in a fixed relationship 
with the second object (11). The computer (36) causes the display of a graphical representation of the position and orientation of die first 
object (12) on a graphical model of previously taken image data of the second object (1 1). 
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SYSTEM FOR LOCATING RELATIVE POSITIONS OF OBJECTS 



This application is a continuation in part of 
applications serial numbers 08/052,042 and 08/052,0457 
5 both filed April 22^ 1993* 

FZEI«D OP THE INVENTION 

This invention relates to locating the position 
of one object relative co another object in a three- 
dimensional space* It more particularly refers to a 
10 system for locating and displaying the position and the^ 

orientation of a first moving object relative to a second 
moving object in three dimensional space. 

Description of Prior Art 

Computed tomography (CT) , magnetic resonance 
15 imaging (MRI) , and other methods provide important 

detailed images of the internals of hximan medical patients 
which are useful for diagnostic purposes. Often, these 
diagnostic tools, which are used to determine and display 
images of the inside of patients' bodies, were used, and 
2 0 the images they create were taken at times other than 

during actual surgical work on the patients, that is 
before and/or sometimes after surgery. Specifically, it 
is usual for CT or MRI scans to be taken before the 
. ^ . surgepn 

25 tools for following an operation in progress. These prior 

scans are then commonly used to plan the surgery or at 
least to assist the surgeon in deciding what surgical 
course of action should be initiated and then followed. 
Sometimes, they are also used after surgery to determine 

30 and evaluate the results of the surgical procedure. 

If surgery is in areas of the body which are not 
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readily visible to the surgeon, such as inside the cranium 
within the recesses of the brain, it is difficult, if not 
impossible to follow the course of the surgery while it is 
going on. It is presently impossible to conduct surgery 
while simultaneously taking an MRI. Therefore, the 
surgeon is always working with previously taken images of 
the internal structures of the patient. 

While working with these previously taken images 
during surgery, there often is no obvious, clear-cut 
relationship between points of interest in the diagnostic 
images and the corresponding points on the actual patient. 
While anomalous tissue may be obviously distinct from 
normal healthy tissue in the images, the difference may 
not be so visible in the patient on the operating table. 
Furthermore, in intracranial surgery, the region of 
interest may not always be accessible to direct view. 
Thus there exists a need for apparatus and systems to help 
a surgeon relate locations in the diagnostic images to the 
corresponding locations in the actual anatomy of the 
patient, and vice versa. 

The related prior art includes references which 
disclose subject matter which describe means to accomplish 
objects which are similar to those of the present 
invention as a whole. These references include 

ptiblications describing the correlation of previously 
taken internal medical images of a patient, which are 
usually three-dimensional, with the corresponding actual, 
present time physical locations on and in the patient in 
the operating room during surgery. U.S. Patent 4,791,934 
describes a semi-automated system which makes this 
lrart^^ti<5n7 ^^^^ 

described in this patent also requires additional 
radiographic imaging to be accomplished in the operating 
room at the time of surgery, and it then also requires 
that these present time images be correlat d into the 
coordinate systems of the previously taken diagnostic 
images so that they can b related to the live patient. 

Furthermore, the system of this '934 patent uses 
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a comput r-driven robot arm to position a surgical tool in 
relation to these images. It do s not measure and define 
the present time location and orientation of an input 
probe (a surgical tool) positioned interactively by the 
5 surgeon, and superimpose such present time data on a 

previously taken image. 

There have been other attempts to solve the 
three-dimensional localization problem specifically for 
stereotactic surgery. One class of solutions has been the 

10 use of a variety of mechanical frames, holders, or 

protractors for surgery (usually intracranial surgery). 
For examples see U.S. Patents 4,931,056, 4,875,478, 
4,841,967, 4,809,694, 4,805,615, 4,723,544, 4,706,665, and 
4,651,732, 4,638,798. Generally these patents disclose 

15 systems which are intended to reproduce angles derived 

from the analysis of internal images, and most require 
rigidly screwing a frame to the skull. In any case, these 
methods are all inconvenient, time-consuming, and prone to 
human error. 

20 A more interactive method is disclosed in United 

States patent 4,750,487. This patent discloses the use of 
present time fluoroscopy in the operating room to help 
guide surgical tools. It will be abundantly clear that 
subjecting a patient to fluoroscopy is undesirable at any 

25 time. It may impose even greater risks during surgery. 

More relevant prior art are the publications 
which describe systems which have been built specifically 
for stereotactic surgery. The following reference is 
illustrative and pertinent: 

30 David W. Roberts, M.D., et al; "A 

. - --r-^.. , T..F^ameless^7. -.T .St^reot^xicw^.x^ ... - 

Computerized Tomographic Imaging and the 
Operating Microscope", J, Neurosurgery 65 , Oct. 
1986. 

35 This reference reports on the use of a three-dimensional 

digitizer to track the position and orientation of the 
field of view of a surgical microscope and means to 
superimpose, on the field of view in the microscope, the 
corresponding internal planar slice of a previously 
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ob-tained computed tomographic (CT) image. Such a system 
was a great step forvard in this art. It was, however, 
not without attendant disadvantages. 

In the first place, the system described by 
5 Roberts et al. used the transmission of sound impulses to 

communicate the location and orientation of the field of 
the surgical microscope. The use of sound is necessarily 
based on the determination of transmission distances as a 
function of the speed of sound, and the differentiation of 

10 small distance differences. It is well known that the 

speed of sound varies to a substantial extent as a 
function of the temperature in the medium through which 
the sound travels. With modern air conditioning and 
heating, there are many different thermoclines and air 

15 currents present in an operating room, which are of little 

concern to a patient or to the doctors , but which can 
materially effect the accurate measurement of precise 
distances as a function of the speed of sound 
transmission. Therefore very accurate compensation 

20 factors must be applied in order to get accurate 

information on the exact location and orientation of the 
operating microscope. It will be appreciated that small 
errors in this field can have very serious consequences to 
the patient • 

25 It must be remembered that the purpose of 

measuring distances from the sound emitters to the 
receivers is to determine precisely where the point of a 
surgical probe, or other device, is. The point may move 
only a very short distance, millimeters or even less, and 

30 so the position of the sound emitter on the probe my 

system must be able to detect such small changes in 
position and orientation of the sound emitters, and to 
translate these small changes into meaningful changes in 
35 the defined position and orientation of th point of the 

probe. Thus the accuracy of measuring and knowing the 
speed of sound in the medium of the operating room is 
critical to locating the point of the probe with any 
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accuracy and cross correlating this location with a 
previously taken image which corresponds to the location 
of the probe tip. It should therefore be clear that the 
major disadvantage for this reported sonic system is the 
5 inherent inaccuracy and instability of the sonic 

mensuration apparatus. 

In addition to the difficulty of making accurate 
and precise distance measurements using sound 
transmission, there was another difficulty and uncertainty 

10 which was inherent in the Roberts et al . system regardless 

of its use of sound sensing means or. not. The Roberts et 
al. system relied on the position of the patient, at least 
so much of the patient as corresponded to the area being 
operated on-that is for example the head, being absolutely 

15 fixed. That system was not capable of determining changes 

in the location of the; patient housing the operating 
microscope, but tracked only the location of the 
microscope independent of the patient. The location and 
orientation of the patient (the head of the patient) was 

20 determined at the start of the operation and the 

previously taken GT scan related to that specific position 
and orientation. If this position and/ or orientation of 
the patient was changed, intentionally or even 
inadvertently, during the entire operation, the system had 

25 to be re-correlated. That is, the operation had to be 

stopped and the previously taken CT scan had to be 
correlated with the new position and/or orientation of the 
patient's head. Clearly this was a major disadvantage of 
the Roberts et al. system. Not only does it require that 

30 the system be re-correlated at the time that the surgeon 

" * " " fie^d r but: it wSs^hb^^ 

capable of taking into consideration inadvertent movements 
of the patient's head during the operation. 

The present invention does not include the 

35 taking of suitable images of the internals of a patient 

before the operation. It starts from the point at which 
these images have air ady been taken and are found to be 
acceptable by the surgeon. This invention therefore does 
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not comprise the imaging apparatus used to generate the 
internal three-dimensional image or model of the internals 
of the patient or other object. However, this invention 
does use these previous imaging data and inputs this 
5 information into the instant system. The system of this 

invention does not include the means of taking these 
images, but ^ it does include the images themselves, 
preferably in electronic form. 

It is contemplated that such known imaging 

10 devices might be ultra-sound, computed tomography (CT) or 

magnetic resonance imaging (MRI) . It is also contemplated 
that such imaging device might be one which has as yet not 
been developed. The important limiting factor in the 
sense of the imager is that the data generated by that 

15 imager must be available in an electronic digital format, 

or is readily convertible into such format. The digital 
data may be derived directly from such an imager and 
transmitted to the instant system over a conventional 
communication network or through magnetic tape or disk 

20 media. 

There is another area of prior art references 
which may be applicable to the patentability of the 
instant invention. This prior art is related specifically 
to localizing devices, which measure the relative 

25 positions of a manually maneuvered probe and another 

object, but not necessarily applied to "seeing" the 
present time position of the probe internal to a patient. 
Previous methods and devices have been utilized to sense 
the position of an object in three-dimensional space. 

30 These known techniques employ various methods of 

Numerous three-dimensional mensuration methods 
project a thin beam or a plane of light onto an object and 
optically sense where the light intersects the object. 
35 Examples of several United States patents which disclose 

such simple distance range-finding devices using this 
general approach are: U.S. patents 4,660,970, 4,701,049, 
4,705,395, 4,709,156, 4,733,969, 4,743,770, 4,753,528, 
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4,761,072, 4,764,016, 4,782,239, and 4,825,091. Examples 
of United States patents which disclose using the plane of 
light to sense an object's shape include: U.S. patents 
4,821,200, 4,701,047, 4,705,401, 4,737,032, 4,745,290, 
5 4,794,262, 4,821,200, 4,743,771, and 4,822,163. In the 

latter, the accuracy of determining the location and 
orientation of the surface sample points is limited by the 
typically low resolution of two-dimensional sensors which 
have usually been- employed (currently the accuracy of 

10 these devices is about 1 part in 512 for a solid state 

video camera).:- Furthermore, these devices are not 
presently capable of detecting the location and 
orientation of the tip of a probe whereby the tip of the 
probe identified is coincident with specific points, and 

15 identifying the location of such probe tip is synonymous 

with identifying such specific point. Additionally, 
optical systems are traditionally limited by line-of-sight 
considerations. If you cannot see the point in question, 
light cannot be directly impinged on that point or 

20 projected from it. Because of these inherent limitations, 

these, known devices have been generally useless for 
locating a point within a recess, which is necessary for 
intracranial surgery. 

The internal imaging devices themselves (such as 

25 computed tomography, magnetic resonance imaging, or 

ultrasonic imaging) are unsuited for tracking the spatial 
location and orientation of a manually held probe during 
an operation, even though they are unencumbered by line- 
of-sight restrictions. Thus, these systems are not 

30 capable of being used to previously record an image, or a 

, . . -^^g iiSalges r of '^Cfie"^ iiiternals'^of a and also to 

image these same internals in present time during an 
operation. 

Other prior art methods and apparatus are known 
35 which track the position of one or more specific moveable 

points in three-dimensional space* In these known 
techniques, the moveable points are generally represented 
by small radiating emitters which move relative to fixed 
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position sensors. Some methods interchange the roles of 
the emitters and sensors. The typical forms of radiation 
are light (U.S. Patent 4,836,778 for example), sound (U.S. 
Patent 3,821,469) , and magnetic fields (U.S. Patent 
5 3, 983, 474), Other methods include mechanical arms or 

cables (U.S. Patent 4>779,212). Some electro-optical 
approaches use a pair of video cameras plus a computer to 
calculate the position of homologous points in a pair of 
stereographic video images (for example, U.S. Patents 
lO 4,836,778 or^ 4,829,373). The points of interest may be 

passive reflectors or flashing light emitters. The use of 
light emitters tend to simplify finding, distinguishing, 
and calculating the location and orientation of the 
points. 

15 Probes with a pointing tip and sonic localizing 

emitters on them have been publicly marketed for several 
years. The instant invention is also concerned with 
determining the location and orientation of a stylus, but 
it is an improvement over the known devices in that it 

20 employs tiny light emitters, in place of the known sound 

emitters. Further, as will become apparent, the method 
used to sense the positions of these light emitters is 
different from what has been used in connection with 
sound . 

25 Additional prior art related to the instant 

invention is found in these references: 

Fuchs, H.; Duran, J.; Johnson, B. ; "Acquisition 
and Modeling of Human Body Form Data", Proc. SPIE, v 166, 
(1978) , p 94-102. 

30 Mesqui, F. ; Kaeser, F.; Fischer, P.; "Real- 

time, Non- invasive Recording and 3-d Display of the 
Functional Movements^ of^a:n^^ ^ Arbitrary^ 
SPIE Biostereometrics 602, 1985, p 77-84. 

Yamashita, Y. ; Suzuki, N. ; oshima, M. ; 
35 "Three-Dimensional Stereometric Measurement 

system Using Optical Scanners, Cylindrical 
Lenses, and Line Sensors, Proc. SPIE, v. 

361, 1983, p. 67-73. 
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The paper by Fuchs, et_al, (1978) best describes 
a method used to track a surgical probe in three- 
dimensional space. This method could possibly be used in 
conjunction with and to supplement the practice of the 
5 instant invention. It is based on using three or more 

one^dimensional sensors, each consisting of a cylindrical 
lens and a linear array of radiation detectors, such as a 
charge-coupled semiconductor device (CCD) or a 
differential-voltage position sensitive detector (PSD) . 

10 The sensors determine intersecting planes which all 

correspond to a single point radiating light emitter. 
Calculation of the point of intersection of the planes 
gives the location of the emitter. The calculation is 
based on the locations, orientations, and other details 

15 concerning the one-dimensional sensors and is a 

straightforward application of analytic geometry. This 
photo-optical method, however, has not been previously 
used for the purpose of the present invention. In that 
sense, it is possible that the instant system could be 

20 considered to be a new and unobvious use of an existing 

system. 

Thus, there still remains a need for a complete 
system (apparatus and method) which provides fast, 
accurate, safe, and convenient mensuration of the three- 

25 dimensional position and orientation of a manually 

operated probe relative to a moveable object of interest. 
This system must also visually relate, in the same 
coordinate system, the relative position and orientation 
of the probe, even a portion of the probe, which is out of 

30 line of sight, to an image of a previously-generated 

threei-dxmenisibnal model oFl:he^ object . 

Objects and Summary of the Invention 

One objective of the present invention is to 
provide means for accurate three-dimensional mensuration 
35 of the relative position and orientation of a moveable 

member with respect to a moveable obj ct. 
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Another object of this invention is to provide 
accurate visual relationship between two objects which are 
each moveable with respect to each other as well as with 
respect to the coordinate system in which these movable 
5 objects reside. 

A further object of this invention is to provide 
accurate spacial relationships between a moving probe and 
a moving surgical patient during an operation in an 
operating room, wherein the probe and the patient are 
10 moving relative to each other as well as relative to a 

fixed location and orientation of the mensuration 
apparatus. 

A still further object of this invention is to 
provide an electro-optical mensuration system which is 

15 inexpensive, easy to use, reliable, and portable, and 

which employs a manually positioned probe, or other 
instrument, at least part of which is not within a line of 
sight of the surgeon, and which further employs a means of 
measuring the otherwise "invisible" position and 

20 orientation of the probe tip. 

Another object of this invention is to provide 
a simple, non-invasive system for establishing a 
correspondence between a presently existing coordinate 
system containing a movable object and a previously 

25 obtained coordinate system containing a three-dimensional 

computer model of that object, where the previously 
obtained computer model is also of this same system. 

Another object of this invention is to relate a 
measured location on the outside, or inside, of an object 

30 to its corresponding location in a previously generated 

- <:bmputer model 6 f tha t^ oto j lect ^ " by e^t abi 1 shing ^ 
correspondence between the coordinate systems of the 
object and the model. 

Another object of this invention is to display 

35 a cut-away view or a cross-sectional slice of a previously 

generated computer model of a planar cross-section of a 
geometric model, where the slice approximately intersects 
the location in the model corresponding to a location 
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measured in present time, and to superimpose a marker on 
the displayed slice to indicate the location on the slice 
corresponding to the measured location. 

Another object of this invention is to assist an 
5 operating surgeon locate subcutaneous diseased tissue 

while avoiding healthy critical structures, especially in 
cranial neurosurgery. 

Additional objects, advantages,. and novel 
features of the invention- shall be set forth, at least in 

10 part in the following description. These will also become 

apparent. to those skilled in the art upon examination of 
the following discussion or may be learned by the practice 
of the invention, . The objects and the advantages of the 
invention may be realized and attained by means of the 

15 instrumentalities and in the combinations particularly 

pointed out in the appended claims. 

To achieve the foregoing and other objects, and 
in accordance with the invention, as embodied and broadly 
described herein, the relative position and orientation of 

20 two or more objects, moving independently of each other in 

space, is determined using the apparatus and the method of 
this invention* As used herein, a two moving-object set, 
which is a simple illustration of the system used in this 
invention, is defined as: 

25 at least one first movable object, which may be a 

hand held probe having an invasive tip, for touching or 
for inserting into a second object; 

at least one second movable object with respect to 
which the first object is movable, or is moving; 

30 a present time coordinate system, that is a 

' ^ ^sy^tem which exi sts^ a the €ime of deteinnining 

the spatial interrelationship of the several moving 
objects, which /includes the second object, and in which 
said second object is moving, or can be moved; 

35 a previously taken predetermined three-dimensional 

geometrical model, suitably a computer generated model, of 
the second object suitably provided in an electronically- 
accessible data base form; 
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a previous time coordinate system, which includes the 
previously taken computer model, that is a coordinate 
system which was determined at the time that the data from 
which the computer generated model were taken; 
5 means to relate the previous time coordinate system 

of the previously taken model of the second object to the 
present time . coordinate system of the second object 
itself; 

at least two first radiation emitters, mounted in 
10 spaced relation to each other on a portion of the first 

object which, during movement of said first object 
, _ relative to said second object, are not obscured from 

receiver (s) adapted to receive the radiation emitted 

thereby; 

15 at least three second, additional radiation emitters, 

dif ferentiatable from each other and from the first 
radiation emitters, which are mounted in fixed 
relationship to, and are movable with, the second object 
and which are not obscured from receiver (s) adapted to 

20 receive the radiation emitted therefrom 

at least three radiation sensors, the positions of 
which are known with respect to a present time fixed 
coordinate system, which are so positioned that they can 
detect the positions of at least two of the first 

25 radiation emitters and at least three of the second 

radiation emitters; 

means to differentiate between the first and second 
radiation emitters sufficient to be able to distinguish 
between the several emitters; 

30 means to control the number of contacts between said 

' " " f irist eradiation emitters a time' 
sufficient to track movement of said first object relative 
to said present time coordinate system in which said 
second object resides; 

35 means to control the number of contacts between said 

second radiation emitters and said sensors per unit of 
time sufficient to track mov ment of said second object 
relative to the present time coordinate system in which it 
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resides without interference with the radiation emitted 
from said first emitters; 

computer means coupled to the first and second 
radiation emitters and to the radiation sensors adapted to 
5 receive data from the sensors and from the emitters; 

means to calculate, based on the data received from 
said emitters and said sensors, the position and 
orientation of each radiation emitter with respect to said 
present time fixed coordinate system; 

10 computer means adapted, to independently determine the 

positions and orientations of the first object and the 
second object relative to . the present time fixed 
coordinate system, whereby determining the position and 
orientation of said first object relative to said second 

15 object, from the computed positions of the radiation 

emitters ; 

computer means adapted to, at a frequency which is 
sufficient to substantially continuously accurately relate 
the present time position and orientation of the first 

20 object to the position and orientation of the model of the 

second object, relate the present time position and 
orientation of the second object with the previously 
obtained model of the second object and to electronically 
cross correlate the model of said second object and the 

25 present time position of said second object so as to form 

one composite image of said model which is consistent with 
the present time position and orientation of said second 
object, and to superimpose the present time position of 
the first object on said consistent image in the present 

30 time coordinate system; and, preferably, display means 

acJapted to display th^ 
location of the first object correctly superimposed on the 
model of the second object by correctly displaying a 
representation of the previously taken image of the second 

35 object which corresponds to the location and orientation 

of the second object in present time relative to the 
display of the first object in present time. 
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The method of this invention relates to the 
operation of the above described apparatus. This method 
will be described in relation to "seeing" the location of 
a point of a surgical probe inside the cranium of a 
5 patient, where the inside of the patient's cranium has 

been previously "seen" on an MRI or a CT scan. In 
carrying out this method, both the head of the patient, 
the second object according to this invention, and the 
surgical probe, the first object according to this 

10- invention, will be moved in illustration of the novel 

operation of this invention. 

The method of this invention, described relative 
to the above described apparatus, includes the steps of: 
at a previous time, taking an MRI or a CT or the 

15 like, which may hereinbelow be referred to sometimes as 

the previous scan, through the patient's head with a 
sufficient number and location of slices as to reveal the 
internal structures of the patient's head. Where there is 
an abnormality, the number and location of the slices 

2 0 should be sufficient to depict that abnormality in at 

least one slice; 

establishing and storing an electronic file of the 
scan in the form of a model of the internals of the 
patient's head, including the abnormality, if there is 

25 one; 

relating that electronic file to a present time 
coordinate system; 

at the present time, as opposed to the previous time 
when the scan was originally taken, at sufficiently 

30 frequent intervals to accurately follow present time 

mo\^^ ^ 
radiation emitters operatively associated with the 
patient's head (the patient's head is the second object in 
the generic description of this invention) ; 

35 at the present time, at sufficiently frequent 

intervals to follow present time movement, computing, from 
the detected positions of these emitters, the moving 
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locations and orientation of the second object relative to 
the predetermined fixed coordinate system; 

electronically adjusting the stored model to display 
a view of that model which corresponds to the computed 
5 present time location and orientation of the moving second 

object in the same present time coordination system; 

at the present time, at sufficiently frequent 
intervals to follow present time movement, detecting the 
locations of at least two of the radiation emitters 
10 operatively associated with the probe, (the probe is the 

first object in the generic description of this 
invention); 

at the present time, computing, from the detected 
positions of these emitters, at sufficiently frequent 

15 intervals to follow present time movement, . the position -JXi . 

andt- orientation of the first object in present time - ^ 
relative to the predetermined fixed coordinate, system; 

at sufficiently frequent intervals to track the ^ - 
movement of the first object relative to the second - 

20 object, determining the positions and orientations of the , 

first object relative to the second object by correlating 
the positions and orientations of the first object . _ 
relative to the predetermined fixed coordinate system with - 
the positions and orientations of the object relative to - 

25 the same predetermined fixed coordinate system; 

determining the correlation between the relative 
position and orientation of the probe with respect to the 
model of the object; and indicating the location of 

the probe with respect to the object by displaying a 

30 representation of the position and orientation of the 

* " " prbbe'^'iTi p^ ' - 

the previously taken scan of the internal structures of 
the cranium. 

In the description of this invention, the 

35 radiation emitters have been located on the first and 

second objects, and the sensors for these radiations have 
been located in fixed positions within the present time 
coordination system. It is considered that this 
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arrangement of these emitters and sensors could readily be 
reversed. That is, the emitters could occupy the fix d 
positions in the present time coordination system and the 
sensors could be located on the first and second objects, 
5 The invention would operate in the same manner with either 

arrangement of sensors and emitters. For convenience, the 
description of this invention has placed the emitters on 
the moving objects and the sensors in fixed positions. 
This arrangement should not be considered to be a 
10 limitation on either the apparatus or method of this 

invention. 

Brief. Description of the Drawings 

The accompanying drawing figures illustrate a 
preferred embodiment of the present invention and, 
15 together with the description, serve to explain the 

principles of the invention. 

Figure lA is a block flow diagram of the optical 
mensuration and correlation system of the present 
invention showing the major components of this system, 
20 except for the means to automatically measure the position 

and orientation of the movable and second object. 

Figure IB is similar to Fig. lA but includes the 
additional radiation emitters which will permit 
automatically measuring the position and orientation of 
25 the second object even during the movement of this second 

object. 

Figure 2 is a perspective view illustrating the 
invention in use by a surgeon performing intracranial 
surgery on a patient, and showing a cursor on a display 
"3t) -- screen that marks the corre^ of 
the probe (the first object) within the image of 
previously obtained model data corresponding to the 
cranium (the second object) . 

Figure 3 is a view of a sample display showing 
35 a position of the tip of the probe superimposed on 

previously obtained model data of an inside slice of a 
cranium and the showing reference points of the second 
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object, as depicted in figure lA, as triangles on the 
patient's skull. 

Figure 4 is a schematic perspective view of a 
sample of one of the one-dimensional photodetectors which 
5 are useful in the practice of the present invention. 

Figure 5 is a graph of the image intensity 
(manifested as a voltage or current) versus locations on 
the photodetector surface for a typical light detector 
which could be used by the radiation supported mensuration 
10 and correlation apparatus of the present invention. 

- Figures 6 and 7 are diagrams of the major steps 
performed by the computer to calculate the position of the 
probe (first object) with respect to the model of the 
inside of the cranium (the second object) and to display 
15 a cross-sectional image slice of the model of the inside 

of the cranium on a suitable display screen, such as a 
computer screen (CRT). 

Figure 8 is a schematic view of radiation beams 
depicting an embodiment of this invention. 
20 Detailed Description of The Preferred Embodiments of This 

Invention 

The radiation mensuration and correlation 
apparatus 10 of the present invention, as applied to a 
medical application, which is illustrative of one use of 

25 this invention, is shown schematically in Figure 1. It 

comprises a hand-held invasive probe 12 (first object) 
housing at least two radiation emitters 14 and 16 mount d 
collinear with one another and with the tip 18 of the 
probe 12. At least three remotely located, one- 

30 dimensional radiation sensors 20, 22, and 24 are mounted 

in fixed, spaced relationship to each other and are 
located at known positions with respect to a predetermined 
fixed coordinate system 80. The radiation sensors 20, 22, 
and 24 sense the radiation widely projected by the 

35 individual emitters 14 and 16 and generate electrical 

output signals from which are deriv d the location of the 
probe emitters 14 and 16 and, consequently the position 
and orientation of the probe tip 18 (which may not be 
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visible to the surgeon because it is within the cranial 
cavity), with respect to the fixed coordinate system 80. 
In addition, where it is desired to determine the moving 
position of the cranium during the operation, the three 
5 sensors 20, 22, and 24 can be programmed to sense and 

derive the locations of other reference emitters 70, 72, 
and 74 on the second object 11 (Figure IB) in the same 
manner as for the probe emitters 14 and 16. The rol of 
these reference emitters on the second object is to 
10 automate the calculation of the relationships between the 

present time coordinate system of the model's image 13 
(Figure 2) of the second object, the coordinate system of 
the sensors, and the local coordinate system of the second 
object itself 11. 

A control unit 30 connected to the moveable 
probe 12 via a data line 2 6 and coupled to the remotely 
located sensors 20, 22, and 24 via data lines 28, 32, and 
34, respectively, synchronizes the sensing of the five 
(exemplary) emitters and the differentiation between them. 
2 0 In that embodiment of this invention where the various 

emitters emit radiation in pulses, as by strobing them, 
the control unit is adapted to control the time 
multiplexing of the two emitters 14 and 16 on the probe 
and the three emitters 70, 72 and 74 on the cranium, 
25 controls the operation of the sensors 20, 22, and 24, and 

receives dif f erentiatable data from these sensors as will 
be more completely described below. A coordinate computer 
36, coupled to the control unit 30 by a data line 38, 
calculates the three-dimensional spatial location of the 
30 probe emitters 14 and 16 and consequently the position and 

" " """""o^i^trsr^^ or ^ the probe tip and'^orr^Iates those 

positions with data from correlation information 4 2 and 
from a model 13 of the second object 11 which has been 
previously stored electronically in an electronically 
35 accessible database 40. Finally, the computer 3 6 causes 

an associated cathode ray tube-monitor (CRT) to display 
the representation of the position and the orientation of 
the probe tip 18 with respect to the computer image 13 of 
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the cranium 11 on display screen 44 (Figure 2) as will be 
more fully described below. 

The probe 12 could be used without the cable 26, 
in that it could be coupled to the control unit 3 0 by 
5 employing distinctive modulation of the light emitters 14 

and 16 instead of sequentially energizing (strobing) them, 
or by varying the wavelength or type of the radiation 
emitted therefrom. For example, the wave forms, color, or 
frequencies of each could be different. In the case of 

10 using sound radiation, the frequencies of the sound 

emitted by the different emitters could be varied so as to 
differentiate between them. The controller 30, by 
detecting the differences between different emitters, that 
is the wave form, color, frequency or other dissimilarity, 

15 of the emitted radiation, can determine to which emitter 

the sensors 20, 22, and 24 are reacting. 

The fundamental mensuration and correlation 
apparatus 10 of the present invention has been illustrated 
in connection with aiding surgeons performing delicate 

20 intracranial surgery. This general use of this apparatus 

does not constitute this invention, nor is it a limitation 
thereon. This use will only serve to illustrate this 
invention. The remaining description continues to use 
such a surgical embodiment as illustrative, although many 

25 other surgical or other applications besides intra-cranial 

surgery are possible (for example, back or sinus surgery 
and breast biopsy) . Moreover, the radiation mensuration 
and correlation apparatus 10 of this invention may be used 
for other purposes in many various medical or non-medical 

30 fields. In the described embodiment, the physical object 

ii of " interest , that ^"^ in the generic 

application of this invention, is the head or cranium of 
a patient, and the model of the cranium is replicated 
using a series of parallel internal image slices (of known 

35 mutual spatial relationship) such as those obtained by 

means of computed tomography (CT) or nuclear magnetic 
resonance imaging (MRI) . These image slices are then 
digitized, forming a three-dimensional computer model of 
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the patient's cranium which is then stored in the 
electronically accessible database 40. 

As shown in Figures lA, IB, 2, and 3 a surgeon 
places the tip 18 of the probe 12, that is the first 
5 object, at any point on or inside the cranium 11 of the 

patient. The position sensors 20, 22, and 24 detect the 
locations of V the emitters 14 and 16 attached to the 
portion of the probe 12 that remains outside the patient's 
body. In order to accomplish this, the radiation produced 

10 by the emitters 14 and 16 must be "visible" to the sensors 

20, 22, and 24. For that reason, more than two emitters 
may be placed on the first object so that the radiation 
from at least two of them will always be visible to the 
sensors. These emitters 14 and 16 are effectively point 

15 sources and radiate energy through a wide angle so that 

this radiation is visible at the sensors over a wide range 
of probe orientations and positions; 

The sensors 20, 22, and 24, the control unit 30, 
and the computer 36 cooperate to determine the three- 

20 dimensional location of each emitter 14 and 16 within a 

coordinate system, and compute the coordinates of each 
emitter in the predetermined fixed coordinate system 80, 
in present time. The computer 36 can then calculate the 
position and orientation of the tip 18 of the probe 12 

25 with respect to the predeteznnained fixed coordinate system 

80, according to the locations of the emitters within the 
fixed coordinate system 80 and the dimensions of the 
probe, which dimensions had been placed into the memory 
(not shown) of the computer 36 beforehand. It should be 

30 noticed that the computer 3 6 can also easily compute 

^ - - "^^si:txort and b Ispecif ic^ 

locations on the probe (such as the vector from emitter 14 
to the tip 18) . Once the computer 36 has calculated the 
location of the probe tip 18 with respect to the fixed 

35 coordinate system 80, the computer 36 then uses the 
relationship between the model of the cranium, which had 
previously been obtained and stored in the database 40, 
and the fixed coordinate system 80 to calculate the 
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position and orientation of the probe tip 18 in relation 
to the model of the second object 11. Finally, the 
computer 36 displays a representation of the model- 
relative position and the orientation of the tip 18 on a 
5 display screen 44. In a simple form of the preferred 

embodiment of this invention, the computer 36 accomplishes 
this display by accessing a previously taken CT or MRI 
image slice: 13 stored in the database 40 that is closest 
to the present time position of the probe tip 18, and then 

10 superimposes a suitable representation 76 of the tip 18 on 

the image 13 as shown in Figures 2 arid 3- Thus, the 
surgeon knows the precise position and orientation of the 
tip 18 in the patient's cranium relative to the image data 
by merely observing the display screen 44. A most 

15 preferred form of the present invention can derive and 

display an arbitrary oblique cross-section through the 
multiple image slices of the MRI, etc, where the cross- 
section can be, for example, perpendicular to the probe 
orientation. 

20 The details of the optical mensuration and 

correlation apparatus 10 of the present invention are best 
understood by reference to Figures 1 and 4 collectively. 
Essentially, the probe 12 supports the two radiation 
emitters 14 and 16, which are rigidly attached to the 

25 probe 12 at fixed, known distances from each other as well 

as from the probe tip. Since only two emitters are used 
here as being representative of the practice of this 
invention, the emitters 14 and 16 should preferably be 
collinear with the tip 18 of the probe 12 so that the 

30 computer 36 can determine uniquely the position and 

oFientatiori of ^ t^^^ three^ dimens 

for reasonable measurement accuracy, the emitters 14 and 
16 should preferably be at least as far from each other as 
the nearest one is from the tip 18. In any case, the 

35 geometrical relationship of the emitters 14 and 16 to each 

other and to the probe tip 18 must be specified to the 
computer 36 beforehand so that the computer 36 can compute 
the exact location of the tip 18 based on the locations of 
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the individual radiation emitters 14 and 16. The use of 
three or more non-collinear emitters would not require 
that any two of them to be collinear with the probe tip. 
Three or more non-collinear emitters would permit the 
5 computer to compute full position and orientation 

information (yaw, pitch, and roll) for the probe. 
Although the, invention is described as showing only a 
cursor locating the relative position of the probe tip 18, 
the invention can be modified to display a line or a 

10 shaped graphic or other icon to indicate the position and 
orientation of the probe 12. This would entail only the 
determination of additional points on the probe in the 
same way that the tip of the probe is located. 

The two radiation emitters 14 and 16, as well as 

15 the additional radiation emitters 70, 72, and 74, can be, 

and preferably are, high intensity light emitting diodes 
(LEDs) , which are preferably coordinated such that the 
emission for any one source is distinguishable from the 
emissions from the other sources. One such 

20 differentiation is to have the emitters time-multiplexed 

or strobed by the control unit 3 0 in a predetermined 
sequence such that only one light emitter is "on" or 
emitting light at any one time. The light emitted from 
any one of these emitters is detected by each of the three 

25 light sensors 20, 22, and 24, which then determines the 

location of each particular emitter in relation to the 
known positions of the sensors 20, 22, and 24 at the time 
it is strobed. 

Each of the one-dimensional sensors 20, 22, and 

30 24 used in the preferred einbodiment 10 of the present 

" ' "Ihveiit ion can be xdehtical "to th^ o^ v 
Therefore, for the purpose of giving a detailed 
description of this embodiment , only the sensor 20 is 
shown and described in detail in figure 4 since the 

35 remaining sensors 22 and 24 are identical. 

In Figure 4, the representative one-dimensional 
sensor 20 comprises a cylindrical lens 46 having a 
longitudinal axis 48 which is orthogonal to the optical 



BNSDCX:iD: <WO 9423647A1 I > 



wo 94/23647 



PCT/US94/04298 



23 

axis 50 of the sensor 20, A linear radiation detector 52, 
such as a charge coupled device (CCD) with several 
thousand elements (or a similar device capable of linear 
positional radiation detection of a suitable "image") , is 
5 positioned in such a manner that the "optical" axis 50 

passes through the center of the aperture 54 of the 
radiation detector 52 and such that the longitudinal axis 
of the aperture 54 is orthogonal to the longitudinal axis 
48 of the lens 46. Radiation ^ such as light beams, 56 

10 from the emitters 14 (and in the~ same manner emitters 16, 

70, 72, and/ or 74) are focused by the cylindrical lens 46 
into a real image line 58 on the surface 60 of linear 
detector 52. 

The detector, illustrated by a photodetector 52, 

15 then generates an output 68 (Figure 5) that is related to 

the position of a real image line 58 on the surface 60 of 
photodetector 52, thus characterizing the location of the 
image itself. That is, those elements or points of the 
photodetector 52 illuminated by the real image line 58 

20 will generate a strong signal, while those not illuminated 
will generate none or very weak signals. Thus, a graph of 
image intensity (or signal strength) versus locations on 
the surface of the photodetector will resemble a signal 
peak curve 68 (see for example Figure 5) . The "all- 

25 emitters-of f " (or background) signal level 66 is never 

quite zero due to the effects of environmental radiation, 
such as light in the operating room, electronic noise, and 
imperfections in the photodetector. In any event, since 
the image of the illuminated emitter is focused into line 

30 58, only the angular displacement of emitter 14 from the 

'^ "^^^ plane of ^^' t^ 

axis 54 is measured by the sensor 52, hence the 
designation "one-dimensional sensor". 

Thus, a single one-dimensional sensor 20 can 

35 only locate the plane on which a radiating emitter 14 

li s. The detector 20 cannot, by itself, deteirmine the 
unique point in space on that plane at which radiating 
emitter 14 is located* To precisely determine the 
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location in space of the radiating emitter 14 requires at 
least three such sensors positioned in spaced relationship 
to each other, since the intersection of three planes 
defined by the three sensors, respectively, are required 
5 to define a single point in space. 

To locate the position of one particular 
radiating emitter, such as 14, the sensors 20, 22, and 24 
are mounted so that the optical axes of their lenses 48 
are not all parallel and no two of such axes are 

10 collinear. In a preferred embodiment of this invention, 

two light sensors, such as sensors 20 and 24 in Figure 2, 
are situated so that their respective axes 48 (Figure 4) 
are in parallel, spaced relationship, and the third 
detector 22 is situated between and equidistant from the 

15 other two detectors, but with its axis 48 perpendicular to 

the axes of the other two. That is, the sensors 20, 22, 
and 24 should be arranged along a line or arc (Figure 2), 
such that each sensor 20, 22, and 24 is generally 
equidistant from the center of the volume in which the 

2 0 measurements are made, equally spaced from each other, and 

all aimed at the center of the measurement volume. 
Suppose for example that the sensors 20, 22, and 24 are 
arranged along a horizontal arc and the optical axes of 
all sensors are oriented horizontally. Then the middle 

25 sensor should be oriented so as to measure the angular 

elevation of the radiation emitters as described above. 
The two outer sensors measure the horizontal angle 
(azimuth) relative to the fixed coordinate system 50. 
Data from the outer sensors are used to stereographically 

30 calculate both the horizontal position and distance from 

~ the sienisors as will: be more ftflly tiescribed belrowv ^ - 

The accuracy of three-dimensional measurement 
depends on the angle formed between the optical axes of 
the outer two sensors 20 and 24, where the emitter to be 

35 measured is at the vertex of the angle. Accuracy will 

improve as that angle approaches a right angle. At least 
three of the several possible sensors 20, 22, and 24 must 
be spaced so that the desired measurement volume is 
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completely within their field of view which can be 
accomplished by making the focal length of the lens 4 6 
short enough to provide coverage of the entire desired 
field of view- In another embodiment of this invention, 
additional sensors, which may be substantially identical 
to sensors 20, 22; and 24, could be used to provide more 
viewpoints, to broaden coverage of the field of view, or 
to enhance measurement accuracy. 

While this process of detecting a given 
radiating emitter, such as 14, can determine the exact 
location of the radiating emitter, it cannot by itself 
determine the particular orientation and position of the 
probe or its tip 18 in three-dimensional space. To do so 
with only two emitters requires that both emitters 14 and 
16 be collinear with the probe tip 18, as described above. 
Also, the distances between each emitter, 14 and 16, and 
the probe tip (as well as the distances between th 
emitters 14 and 16 themselves) must be known and, loaded 
into the memory of the computer 3 6 before the computer 36 
can determine the position and orientation of the probe 
tip 18 from the locations of the emitters 14 and 16 in the 
fixed coordinate system 80. Consequently, when each of 
the radiation emitters 14 and 16 is rapidly turned on in 
sequence, or strobed, the sensors 20, 22, and 24 can 
detect the exact location of each emitter in turn. Thus 
computer 36 can determine the exact position and 
orientation of the probe, and therefore its tip 18. Since 
only one of the radiation emitters 14 or 16 is on at any 
one time, the detectors 20, 22, and 24 locate the location 
of that particular illximinated emitter individually. If 
the "^ strobe ^^ "^^ at "which^ the 

emitters 14 and 16 are turned on and off in sequence, is 
fast enough, the detectors 20, 22, and 24 can, for all 
practical purposes, determine the position and orientation 
of the probe 12 and its tip 18 at any instant in time, and 
therefore can follow the movement of the probe tip in 
present time, that is during the time that the probe tip 
is actually moving. In other words, this system can 
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simulate the movement of the probe tip on the previously 
taken image in present time during the surgical procedure. 

The sensors 20, 22, and 24 need only distinguish 
which of the radiation emitters 14, 16, 70, 72, or 74 is 
5 on at any one time, in the preferred embodiment 10 of the 

present invention , this function is accompl ished by 
strobing each of the emitters in sequence, as described 
above. However, other methods can be used to allow the 
sensors 20, 22, and 24 to distinguish the respective 

10 radiation emitters 14, 16, 70, 72, and 74 from one 

another. For example, different wave lengths <colors) of 
light, or different frequencies of sound, could be used in 
conjunction with detectors capable of distinguishing those 
particular different radiations. 

15 Alternatively, it is one aspect of this 

invention to modulate each of the respective radiation 
emitters 14, 16, 70, 72, and 74 with a unique wave form or 
pulse train. This means of differentiating between the 
different emitters is believed to be novel and unique to 

20 the instant invention. If such different wave forms or 

pulse trains are used to differentiate the different 
emitters, it is also within the spirit and scope of this 
invention to transmit additional information on these wav 
forms, such as for example the temperature of the 

25 particular structure being contacted by the probe tip 18. 

It is within the scope of this invention to provide means 
readily accessible to the surgeon or other operator to 
engage or disengage the taking of such additional 
information and the transmission thereof by the unique 

30 wave forms or pulse trains radiated by the respective 

" "^^"'^ ^nrrtitters V """ "Under these tsxrc uh i t 

3 0 or computer 36 will be designed to demodulate the wave 
form to determine to which particular emitter the sensed 
signal belongs, and to decode the additional information 

35 being transmitted; 

Numerous other methods for distinguishing the 
radiation emitters ar possible. Therefore, the present 
invention should not be regarded as limited to the 
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particular strobing method shown and described herein, but 
is generic to the use of any means to differentiate 
between the different emitters. 

Conventional or unique auto-focusing or 
5 multiple-lens radiation detection may be integrated into 

the sensors 20, 22, and 24 to improve the performance of 
the system. However, the simple, fixed-focus optics shown 
and described herein and shown in Figure 4 for one sensor 
provide a good level of performance if the working range 

10 of the probe is restricted. Even if the real image of an 

emitter, such as 14, is somewhat out of focus on the 
detector 52, the angular measurement of the image is still 
usable. A usable measurement for each of the sensors 20, 
22, or 24 to generate will be any of the following: (1) 

15 the position of the detector element with peak intensity, 

(2) the intensity -weighted average (centroid) of all over- 
threshold, elements, or simply (3) the average of the 
minimum and maximum elements where the intensity is over 
some threshold. The detector 52 should be placed at the 

20 focal distance for the farthest typical operating distance 

of the radiation emitters. Closer emitters will form 
slightly defocused images 58, but they require, less 
precise angular measurement for a given distance accuracy. 
Furthermore, their de-focused real images are brighter, 

25 which increases the brightness gradient at the edges of 

the image • 

As described so far, the real image 58 of the 
currently activated emitter must be significantly 
different from (for example brighter than) the rest of the 
30 radiation falling on the sensor 52. Otherwise, other 

^ - ' ref 1 ective^ surf aces" in "the field the 
sensors will hinder the detection of the emitter's real 
image. Therefore, it is desirable to include in the 
apparatus, circuitry to subtract the background radiation 
35 received by the sensors from other, ambient, sources. 

This per se known circuitry enhances use of the 
invention where the sensors are required to detect the 
radiation emitt rs against relatively bright backgrounds. 
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While the radiation emitters are all momentarily 
extinguished, the one-dimensional data from each sensor 
are saved in a memory. This can be done in an analog 
delay line or by digitally sampling the output signal and 
5 storing it in a digital memory* Then, as each emitter is 

"viewed" sequentially, the saved data are subtracted from 
the current data generated by the currently radiating 
emitter. If the background data are stored digitally, the 
current data are also digitized, and the stored background 

10 data are digitally subtracted from the current data. 

A graphical representation of the radiation 
intensity of the image or, equivalehtly , the generated 
output voltage amplitude for each element in a row of 
detecting elements, is shown in Figure 5. The graph 

15 depicts typical background image intensities 66 with all 

emitters off, the intensities 68 with one radiation 
emitter on, and the element-by-element difference 64 
between the intensities with the emitter off and those 
with it on. The measurements will likely contain some 

20 random noise, electronic or otherwise, and two consecutive 

measurements for a given sensor element may differ 
slightly even where the background is unchanged. 
Therefore, the differential intensities 64 between two 
consecutive measurements also contain some random 

25 electronic noise. However, the two measurements differ 

substantially only at the location of the radiation 
emitter image, and this difference exceeds the threshold 
level 62 . 

The details of the structure and operation of 
30 the control unit 30 are best seen in Figure 6. 

^pecrt* icany control unlt^ 3 O" fse^ Figure lA^ and IB ) " 
supplies power to the radiation emitters 14, 16, 70, 72, 
and 74 and the radiation sensors 20, 22, and 24. A 
control and synchronization unit 84 and radiation source 
35 sequencer 88 (wher a strobed radiation sequencing is 

used) time-multiplexes or strobes the radiation emitters 
individually, as described above, so that the position and 
orientation of the probe tip 18 (Figure l) can be 
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determined from the signals received from the sensors 20, 
22, and 24* The angular data signals received from the 
sensors 20, 22, and 24 are converted by an analog-to- 
digital converter 92. Actually, three analog-to-digital 
5 converters are used, as shown in Figure 6, but only one is 

labeled and described herein for brevity, since the other 
two analog^to-digital converters are siibstantially 
identical and are used to convert the signals from the 
other sensors 22 and 24. 

10 The control and synchronization unit 84 also 

controls three switches, of which switch 93 is typical, 
which store all digital data received from the sensors 20, 
22, and 24 when the radiation emitters 14 and 16 aire off 
and stores these data into a background memoiry 94. Then, 

15 when the radiation emitters 14, 16, 70, 72, and 74 are 

illuminated in sequence by radiation source sequencer 18, 
the synchronization and control unit 84 change's the state 
of switch 93 which then redirects the data from the three 
sensors 20, 22, and 24 to a subtraction unit 91. The 

20 subtraction unit 91 subtracts the background data from the 
emitter radiation data, thus resulting in a signal which 
has been relatively freed from the background signal 66 
(Figure 5) since the fixed pattern noise has been 
subtracted from the signal. 

25 As shown in Figure 6, which should be considered 

in conjunction with Figure 5, a 1-D (one-dimensional) 
position calculation unit 95 determines the location of 
the real image line 58 on the CCD sensor 52 (Figure 4) by 
measuring the locations of the edges 67 and 69 of the 

30 signal blip 68 (Figure 5) generated by the CCD sensor 

'^"'''''^ ""based^ o^ "The 
1-D position calculation unit 95 then averages the 
distance between the two edges to find the center of the 
signal peak 68 as shown in Figure 5. This method of 

35 determining the center of the signal p ak is per se well 

known in the art and need not be described in further 
detail. Moreover, numerous other methods of determining 
the location of the signal peak or its centroid are known 
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in the art and will be obvious to those of ordinary skill 
in th art. The method used depends on the signal 
characteristics of the radiation sensor used as well as 
the characteristics of the lens system used to focus the 
5 radiation onto the surface of the detector, in addition to 

other parameters. Those practicing this invention with 
the various alternatives described herein would have no 
trouble selecting a signal detection algorithm best suited 
to the particular characteristics of the sensors and the 

10 particular radiation being used. 

Finally, the control unit 30 (Figure 1) 
transmits the radiation data to the computer 36. That is, 
when the computer 3 6 is ready to compute the current 
location of the currently radiating emitter, such as 14, 

15 the latest angular data from all sensors 20, 22, and 24 

are provided for analysis. If the sensors generate data 
faster than the control unit 30 can process them, the 
surplus angular data are simply discarded. 

The operation of the computer 36 is most 

20 advantageously set forth in Figure 7. The computer 36 

calculates one-dimensional positions for each radiation 
emitter such as 14 or 16, based on the location of the 
signal peak from each respective sensor 20, 22, and 24. 
These one-dimensional angular position measurements are 

25 then used to determine the three-dimensional spatial 

coordinates of the emitters 14 and 16 and thus for the 
position and orientation of the probe 12 relative to the 
predetermined fixed coordinate system 80 by coordinat 
transformation methods which are per se well-known in the 

30 art. The output signals from the computer 36 can be in 

. smy^toTmd&s±rB<^^^ the 
application system, such as XYZ coordinate triples based 
upon the predetermined fixed coordinate system 80. 

Figure 8 and the following paragraphs describe 

35 in detail how the location of a single radiation emitt r, 

such as 14, is computed from the data derived from the 
sensors 20, 22, and 24. The following description applies 
to these three sensors 20, 22, and 24 only. If there are 
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more than three such sensors, the calculation can b 
performed using any three or more of the sensors. 
Furthermore, if more than three sensors are used, the 
average of the points calculated from all combinations of 
5 three sensors could be used to increase accuracy. Another 

option is to use the point calculated from the three 
sensors closest to the radiation emitter 14 or 16. Th 
following parameters are considered to be known XYZ 
constants: 

10 DO[i], one endpoint of each linear photodetector i; 

bl[i], the other endpoint of linear photodetector i; 
LO[i], one endpoint of the axis of each lens i; and 
Ll[i], the other endpoint of the axis of lens i. 
Each sensor generates T[i], a parametric value between 0 

15 and 1 indicating where the peak or center of the line 

image of the emitter intersects the line segment betw en 
DO[i] and Dl[i]. The XYZ coordinates of point S are to be 
calculated, where S is the location of the radiation 
emitter. For a CCD radiation detector array, T[i] is the 

20 index of the element on which the center or peak of the 

image falls divided by the number of elements on the 
detector array. 

The three-dimensional coordinates of the above 
points are all referenced to a predetermined fixed 

25 coordinate system 80. The cylindrical lens and linear 

photodetector do not directly measure the angle A of the 
radiation emitter about its lens axis ; rather , they 
measure a value T[i] linearly related to the tangent of 
that angle: 

where C is a constant of proportionality that is related 
to, and determined empirically by, the dimensions of a 
particular system. 

The three-dimensional location of the image line 
3 5 on the linear photodetector is: 
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D[i] = (1 - T[i]) * DO[i] + (T[i]) * Dlti] 

If the lens is ideal, then S also lies in plane 
P[i]. In reality, the point D[i] might have to be 
computed by- a non-linear function F(t) that corrects for 
5 non-linear aberrations of the lens or the photodetector : 

D[i] = (1 - F(T[i])) * DO[i] + (F(T[i])) * Dr[i] 

Function F(t) could be a polynomial in variable 
or it could be a value interpolated from an empirically 
determined table. 

10 P[i] is the unique plane determined by the three 

points D[i3, LO[i], and Ll[i], which are never collinear. 
S is the point of intersection of the planes P[l], P[2], 
and P[3] determined respectively by sensors 1, 2, and 3. 
S is a unique point if at least two sensor lenses 

15 longitudinal axes 48 are not parallel and if no two lens 

axes 48 are collinear* The intersection point is found by 
finding the common solution S of the three equations 
defining the planes P[i3. Once the location S of each of 
the probe's radiation emitters is computed, the location 

20 of the probe's tip 18 can be calculated. The method of 

making such a detearmination is well known using the 
teaching of analytic geometry and matrix manipulations. 

If M is a linear transformation describing the 
relationship between a point R in the image coordinate 

25 system and a point S in the fixed coordinate system, then: 

R * M = S. 

If M""* is the inverse of M and if S is a point in the 
fixed coordinate system, then the point R in the image 
coordinate system corresponding to S is: 

30 S * M'V = R. 

Now, suppose that the second object is moved in the 
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mensuration coordinate system. This can be described by 
a linear transformation U wher the coordinates S of a 
point are mapped to the coordinates S ' : 

S * U = S' 

5 Then the old value- of M above must be multiplied by U in 

order to correct the relationship between the point R in 
the image coordinate and the corresponding point in the 
mensuration coordinate system because of the relative 
movement of first object with respect to the second 
10 object: 

R = S' * U"^ * M'^ 

The preliminary- steps required before practicing the 
method of the invention are now described. Then, after 
fully describing these preliminary steps, the detailed 
steps of the method of the optical mensuration and 
correlation apparatus are described* 

Use of the invention takes place in three 
phases: the imaging phase, the correlation phase, and the 
normal operation phase. The imaging phase precedes the 
normal operation of the present invention. During the 
imaging phase, a scan of the body of the second object of 
interest is used to build a three-dimensional geometrical 
model. In the preceding description, the second object 
was the head of a human intracranial surgical patient 
because the invention is advantageously used in 
stereotactic neurosurgery. Accordingly, the three- 
dimenffionai-" model comprises^ xiigital data^^^ f ram ^ a ' ser ies of 
internal cross-sectional images obtained from computed 
tomography (CT) , magnetic resonance (MRI) , ultrasound, or 
some other diagnostic medical scanner. In any case, the 
image data are stored in a suitable, electronic memory 40 
which can be accessed later by the computer 36. The data 
are considered to be stored as a series of parallel two- 
dimensional rectangular arrays of picture elements 
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(pixels) , each pixel being an integer representing 
relative density. If the object is relatively rigid, like 
a human head, this three-dimensional model may be created 
at some time before the correlation and operational phases 
5 of the invention and possibly at another location. 

Also, during the imaging phase, at least three 
non-coil inear reference points 71, 73, and 75 (Figures 2 
and 3) must be identified relative to the object 11. 
These may be represented by ink spots , tattoos , radiopaqu 

10 beads, well-defined rigid anatomical landmarks, locations 

on a stereotactic frame, sterile^ pins temporarily inserted 
into rigid tissue or bone of a surgical patient, or some 
other reference means. The coordinates of these reference 
points are measured and recorded relative to the 

15 coordinate system of the imaging device. One way to 

accomplish this is to capture the reference points as part 
of the previously made three dimensional model itself. 
For example, radiopaque pins could be placed within the 
image planes of diagnostic CT slices; the pin locations, 

20 if not automatically detectable from their high density, 

can be identified interactively by the surgeon using a 
cursor on the computer display of the CT slices. See 
Figure 3. 

The initializing of the position and the 
25 orientation of the second object, the patient's cranium, 

is well known in this art- The instant invention departs 
from this well known operation to add radiation emitters 
which have a known and exact spacial relation to these 
fiducial markings. These additional radiation emitters 
3 0 must then be programmed or otherwise activated for use in 

^ ar part icuXar manner in order^ ta practice ^he^ ^^nst ant 
invention. They must be programmed to radiate at some 
frequency during the surgical procedure which is in 
progress during present time so that the position and 
35 orientation of the second object will be available to the 

surgeon at all relevant times and so that this position 
and orientation can be repeatedly and automatically 
updated in order for the system to revise the specific 
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selected scan slice to superimpose the position and the 
orientation of the first object on in a correct 
depictation of the actual relative positions and 
orientations of both the first and second object in 
5 present time during the surgical procedure • 

The initial correlation mode immediately 
precedes the normal operational phase of the present 
invention and must take place in the operating room. 
During this initial correlation phase, the instant system 

10 accesses the data of the three-dimensional geometrical 

model of the patient (or other object), including the 
reference point (fiducial marker) coordinates which were 
recorded earlier, that is ^previously. Next, the surgeon 
may place the tip of the probe 18 at each of the reference 

15 points 71, 73, and 75 on the patient, in turn. This 

sequence of operations may be directed by the computer 
program. In the alternative, the system of this invention 
provides these data automatically by the radiation- from 
the emitters 70, 72, and 74 being received by the sensors 

20 directly and automatically without special intervention by 
the surgeon. Either of these procedures establish an 
initial relationship between the locations of these 
reference points in the model coordinate system and their 
current physical locations in the fixed coordinate system 

25 80. However, the preferred determination of this initial 

position and orientation also carries on during the whole 
of the surgical procedure and therefore is capable of 
substantially continuously updating the position and 
orientation of the second object and relating it to the 

30 current position and orientation of the probe. In turn, 

■r.. .r^. -^.:u:.^^-^^..^^^^^^::^^^ Txh^Bat lc^^^ relatibSship 

between all points in the model and points in the 
coordinate system 80. Thereafter, when the patient is 
moved relative to the sensors, the prior art must 
35 establish a new relationship by again digitizing the 

ref rence points 71, 73, and 75 within the coordinate 
system 80. That is, the correlation phase must be 
repeated. Again, the system of this invention uses the 
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emitters 70, 72 and 74 to accomplish this automatically. 
For this reason, the automatic tracking of the position of 
the head, or the second object whatever that is, which is 
described below and which overcomes this problem, is an 
5 essential, significant feature of the present invention. 

Since the position and orientation of the head 
is initially ^ and sxibstantially continually thereafter 
correlated with the model, the surgeon can relate any 
locations of interest on the diagnostic images with the 

10 corresponding physical locations on this patient during 

the operation, and vice versa. These include locations 
accessible to the probe tip 18 but not necessarily 
directly visible to the surgeon. 

Having described the function and purpose of the 

15 preliminary steps, the detailed method of the present 

invention is more easily understood. As shown in Figure 
7, the position data 21 of the probe emitters generated by 
the sensors and control unit are converted into three- 
dimensional coordinates relative to the predetermined 

20 fixed coordinate system 80 of the sensors. Using 

dimensional parameters describing the relationship among 
the probe emitters and the probe tip, the computer 
determines the coordinates of the probe tip in a step 39. 
During the initial correlation phase, the probe tip may be 

25 placed at each of the reference points 71, 73, and 75 in 

turn. Alternatively, in accord with a preferred aspect of 
this invention, the emitters 71, 73 and 75 are located by 
the sensors and the correct position and orientation of 
the second object is thereby determined. The coordinates 

30 of the second object in the fixed coordinate system along 

" " " "V±th ""theiir co^ordinateisr 46'^±n ^he "^image coordinate: system 
determine a unique linear transformation relating the two 
coordinate systems in a step 45. This is a per se known 
calculation in analytic geometry and matrix mathematics. 

3 5 As noted above, a more automated and direct 

method of determining the location of the second object is 
to directly read the locations of the fiducial points 71, 
73 and 75 by the fixed sensors 20, 22 and 24. This can be 
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accomplished by placing radiation emitters 70, 72, and 74 
(Figure IB) at those reference points (or in a known fixed 
spacial relationship to those reference points 71, 73, and 
75). The emissions of , these emitters can then be read 
5 directly by the sensors 20, 22, and 24, and thus the 

computer can then automatically determine their locations 
relative to the predetermined fixed, coordinate system 8 0 
of the sensors. Thus the position and orientation of the 
second object, the cranium in the preferred embodiment of 

10 this, invention, can be automatically and substantially 

continuously determined. With the position and 

orientation of the second object being at least 
frequently, if not substantially continuously, updated, 
the position of the first object, the probe, which is also 

15 determined- at least frequently, if not substantially 

continuously, can then be updated in relation ,to the 
second object at the same frequency. Since the position 
and orientation of both the first and the second objects 
are each at least frequently determined and . updated in 

20 relation to the fixed coordinate system 80, the position 

and orientation of each of these first and second objects 
can then" be determined relative to each other, by 
indirect, but well known, calculations which are -easily 
carried out in short order by a computer. 

25 It has been stated herein that the position and 

orientation of the second object, the cranium, can, 
according to this invention, be determined continuously or 
at least frequently. The frequency at which the position 
and orientation of the second object is determined is a 

30 function of the desires of the operator of this system and 

* " ' ^ ^ ^ the^^^f requency " a€' wjiicli ' the " radia t loh^^ and the 

sensors can be operated. In the case where the emitters 
all emit the same wave length of radiation and the sensors 
all sense this same wave length of radiation, the 

35 differentiation of the emissions of th several emitters 

is followed in a sequential pattern. Thus, in this 
embodiment of this invention, the emitters will emit 
radiation in sequence, for example 14, then 16, then 70, 
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"then 72 and then 74. The sensors will have been 
programmed to identify a signal with an emitter as a 
function of when the signal is received. 

In this embodiment of this invention, the 
5 position and orientation of the first object, the prob , 

is determined with the same frequency as is the location 
and the orientation of the second object, the cranixim, 
because all of the emitters radiate in sequence. However, 
the system can be programmed so that the emitters 14 and 

10 16 fire more or less frequently than the emitters 70, 72 

and 74; Under these conditions, the position and 
orientation of the first object and of the second object 
will be determined at different individual frequencies, 
that is at the same frequency as the frequency of the 

15 radiation from their respective emitters. It will 

therefore be clear that the frequency of determination of 
the location of any given emitter, and therefore the 
determination of the position and orientation of these 
first and second objects, is, because of the instant 

20 invented system, for the first time entirely controllable 

by the programmer or the operator, within the capabilities 
of the system operating the emitters. 

However, it should be understood that the 
position and orientation of the first and/or second 

25 objects can be determined in a substantially continuous 

manner . In this embodiment of this invention , each 
emitter will radiate a different wave length or wave form 
or frequency pulse of radiation. Therefore, the radiation 
emitted from each emitter is simultaneously distinct from 

3 0 the radiation emitted from the other emitters. Under 

'"^ ^ " tHes^^^ 

determined continuously by a set of sensors which is tuned 
to the specific, different radiation of each emitter. 
Therefore, the location of each emitter can be determined 
35 continuously, whereby the position and orientation of 

either or both of the objects can be calculated by the 
computer from these continuous locations of the different 
emitters. 
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Whil it is a significant distinction of this 
invention from the prior art that: 
in the prior art: 

the first object is intended to be moved and the 
5 second object is intended to be stationary; and fa 

position and orientation of the first object is frequently 
determined, but the position and orientation of the second 
object is only determined at the start of the operation 
and at any time that the second object, the cranium, which 
10 is intended not to be moved at all during the operation, 

is known by the surgeon to be moved; 
whereas 

according to this invention: 

the first object is intended to be moved, and 
15 the second object is not intended to be rigidly 

immobilized in place, or, put another way, the second 
object is permitted to move and is even expected to move; 
and 

the position and the orientation of the first 

20 object is frequently determined, and the position and 

orientation of the second object is also frequently 
determined. The position and orientation of these two 
ob j ects may be determined at the same frequency ' or at 
different frequencies (or even continuously) as desired by 

25 the operator. 

,In preferred embodiments of this invention, the 
position and orientation of the second object will be 
determined from one hundredth to ten times, most 
preferably from a quarter as often to four times, as often 

30 as the frequency at which the position and orientation of 

the f irst bS j ect is det erin i^ As a general propois it ion 
there is no limit on the relationship between these 
frequencies of measurement. The preferred relationships 
set forth herein are illustrative and not limiting. The 

35 frequency of each measurement is dependent on the amount 

of movement which is allowed and is intended to be shown 
on the CRT. The upper limit on this frequency is 
determined by the" ability of the emitters to be 
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distinguished. There is no lower limit. 

In the instant specification, the emitters have 
been described as being on the first and second objects 
and being movable therewith, and the sensors have been 
5 described as being in a fixed relation to the coordinate 

system. While this is the preferred system, it is by no 
means the only configuration of the system of this 
invention. It is also within the scope of this invention 
to provide the emitters in fixed relationship to the 

10 coordinate system, and the sensors on the first and second 

objects, respectively. The wiring may be somewhat more 
cumbersome in this configuration, but that should not 
detract from the viability of such a reversal. 

This invention has been described with ref erenc 

15 to a first and a second moving object, and the 

determination of each of their absolute and relative 
positions and orientations in a fixed coordinate system. 
Xt will be clear that this same system applies to more 
than two objects. In fact, the position and orientation 

20 of any number of objects can be determined, both 

absolutely with respect to the coordination system, and 
relatively with respect to each other, by the practice of 
this invention. Thus, when this specification and the 
claims appended hereto speak of a first and a second 

25 object, these can be two out of any number of total 

objects. This number is merely illustrative of the 
practice of this invention and is in no way limiting 
thereon. 

Thus, the preferred system of this invention 
30 performs the three primary tasks of this invention, 

" ^ ^ " pref ereiblyv b^tt^ not^ Tiecessariiyv""^simuitaneously r^-^ 

the absolute position and orientation of the second 
object, the cranium, in the fixed coordination system is 
determined at least very frequently; 
3 5 the relationship between the absolute position and 

orientation of the second object with respect to the 
previously taken images of that object, particularly the 
inside structures of that object, is determined at least 
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very frequ ntly; and 

the absolute position and orientation of the first 
object, the probe, is determined at least very frequently. 
The accomplishment of these three tasks then permits the 
5 computer to accomplish the three essential secondary tasks 

of this invention: 

to calculate the position and orientation of the 
first object, the probe, in relation to the second object, 
the cranium, even though the first object, or a portion of 
10 it, is out of the line of sight of either the surgeon or 

the sensors; 

to select the appropriate slice of the previously 
taken model of the interior of the second object which 
corresponds to the present time position and orientation 

15 of the first object in relation to the present time 

position and orientation of the second object; and 

to display the appropriate slice of the previously 
taken image of the second object with the present time 
position and orientation of the first object correctly 

20 depicted thereon. 

Both the initial and the continual correlation 
determinations can be automatically initiated and updated 
by the computer 3 6 in some predetermined timed sequence or 
continuously. In fact, according to the most preferred 

25 aspect of this embodiment of this invention, the 

correlation phase is frequently, briefly from time to 
time, or even continuously, repeated, interspersed in 
between measurements in the operational phase or conducted 
simultaneously with the operational phase of the practice 

30 of this invention for the purpose of recalculating the 

(such as a surgical patient) moves relative to the 
sensors. 

During normal operation, the tip coordinates are 
35 transformed in a step 44 using the transformation computed 

in step 45. The new transformed coordinates, relative to 
the image coordinate syst m, are used to determine the 
plane of some two-dimensional cross-section through the 
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three-dimensional image model 41 accessible in the 
accessible memory 43. The simplest method is simply to 
choose the existing diagnostic image plane located closest 
to the probe tip's coordinates relative to the model 
5 coordinate system. 

In any case, a step 47 transforms the two- 
dimensional cross-sectional slice to a screen image and 
places a cursor on it to mark the location of the probe 
tip superimposed in the image. Scaling and viewing 

10 parameters determine how the image is displayed. Because 

the surgeon may not be able to simultaneously view the 
patient (object) and the computer display screen, the step 
47 should be controlled by the surgeon, such as for 
example by placing an activating button on the probe. 

15 Pressing the button, can be the signal for freezing the 

image and the depicted position and orientation of the 
probe tip marker at that instant on the display screen. 

In a more complex embodiment of this invention, 
the computer system could generate and display on the 

2 0 screen a cut-away view at an arbitrary angle, for example, 

perpendicular to the direction the probe is pointing, 
using the data from multiple image slices. In simpler 
cases, the computer simply displays any one or more 
convenient image slices through the location of the probe 
25 tip. For example, the displayed slice might simply be the 

original CT slice which includes the location of the probe 
tip, or is closest to that location. In any case, the 
computer then causes the image a cursor at the current 
position of the probe tip to be displayed on this 

3 0 previously taken image of a slice through the second 
""""''■"■^""^^■^^"^"*<:)bjiectv^'"""""^ 

An alternative means, to record the location of 
the reference points in the coordinate space of the 
imaging apparatus during the imaging phase, employs an 
3 5 additional, separate instance of the three-dimensional 

position mensuration probe, sensors, control unit, and 
computer of the present invention. In order to implement 
this embodiment of this invention, the additional sensors 
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are permanently attached directly on the imaging 
apparatus. The additional probe measures the location of 
the reference points at the time of imaging, and the 
additional control unit and computer determines and 
5 records their locations relative to the coordinate system 

of the imaging apparatus. The advantage of this approach 
is that the fiducial markers, that is the landmarks or 
reference pins, * need not be within the limited cross- 
sectional slices visible to the imaging device. 

10 As an alternative to „true three-dimensional 

images, standard x-ray radiographs from several distinct 
directions can be used to construct a crude model in lieu 
of the imaging phase described above. Radiographs from 
two or more directions are digitally scanned, and four 

15 non-coplanar reference points on them are identified with 

a cursor or light pen. In a correlation phase similar to 
that described above, these four points on the patient are 
digitized just prior to surgery. Then, during surgery, 
the location of the probe tip is projected onto the 

20 digitized computer images of the two-dimensional 

radiographs where the projection is uniquely defined by 
mapping and transferring the reference point coordinates 
from the model coordinate system to the fixed sensor 
coordinate system. 

25 In a further embodiment of this invention, a 

videotape recording of the computer screen (as well as the 
direct view of the surgeon and patient) is used to help 
document the performance of the instant procedure. 
Radiation emitters may be present on more than one 

30 standard surgical tool such as the microscope, scalpel, 

- - f orceps^^ aiS^ cauteri zer r'eacK^ 

effect, a probe. These emitters should be differentiated 
from each other in the same manner as aforesaid. 

The method and apparatus of the optical 

35 mensuration and correlation apparatus 10 of the present 

invention has been completely described. VJhile some of 
the numerous modifications and equivalents of the system 
of this invention have been described herein, still other 
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modifications and changes will readily occur to those of 
ordinary skill in the art. For instance, the preferred 
embodiment described herein uses visible light, since 
human operators can readily observe if the light sources 
5 are operative or whether they are causing troublesome 

reflections. Clearly, other wavelengths of 

electromagnetic radiation could be used without departing 
from the spirit and scope of the invention. Non-visible 
light, such as infrared or ultra-violet light, would have 

la the advantage of not distracting the surgeon with flashing 

lights. Ultra-sound could be used conveniently. Other 
modifications to the detector "optics" and lenses are 
possible which would change, and possibly improve, the 
image characteristics on the detectors. For example, 

15 toroidal lenses could be used which are longitudinally 

curved along an arc with a radius equal to the focal 
length of the lens. Similarly, the surfaces of the 
photodetectors could also be curved, thus allowing the 
images of distant light sources to remain in sharp focus, 

20 regardless of their positions. Numerous enhancements of 

the digital data are possible by suitably programming the 
computer. 

The most preferred aspects of this invention use 
electromagnetic radiation, and especially visible light, 

25 as the radiation from the emitters. This use of light for 

this function is a major improvement over the use in the 
prior art of audible sound emitters and detectors. 
However, prior art systems which are based on the use of 
sound emitters can be reprogrammed to carry out the 

3 0 operations to substantially continuously recorrelate the 

positionr an^^^ "^obj^ct^during the 

surgical procedure, as they have been described herein. 
Thus, the movement of the second object can be at least 
frequently, if not continuously, tracked using sound 

35 emitters and detectors and suitable temperature 

compensation techniques. In this last regard, the 
aforementioned ability of the instant system to determine 
and transmit the temperature of the probe tip can be used 
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to good advantage when using sound as the radiation of 
choice. The fact that the use of electromagnetic 
radiation, particularly light, emitters and sensors is an 
improvement over the use of audible sound emitters and 
5 sensors is not intended to be a limitation of the practice 

of continuously or frequently following the movement of 
the first or the second objects. That is an invention in 
and of itself using any emitter-sensor pair. 

It should be understood ^ however, that the 

10 transmission between emitters and sensors operate 

differently, and measure different things when 
electromagnetic radiation is used as compared to the use 
of sound, audible or ultrasonic. In the case of 
electromagnetic radiation, what is being measured is the 

15 angle that the radiation path makes between the emitter 

and the sensor relative to some arbitrary fixed line. By 
measuring all of these angles of the rays between the 
emitters and the sensors, conventional analytic geometjry 
solutions will locate the points in space where th 

20 various emitters are. On the other hand, when sound 

radiation is used, what is being measured is the distance 
between each of the emitters and the sensors. Again, 
conventional analytic geometry solutions will precisely 
locate the point in space which is occupied by each 

25 emitters/ sensor. While the casual observer, or the 

operator of the systems of this invention will not observe 
any difference in result, there is a marked difference in 
the way that result is achieved, and therefore this will 
necessitate a difference in the manner in which this 

30 system is programmed. 

of the invention. Since numerous modifications and 
changes will readily occur to those of ordinary skill in 
the art, given the teachings of this specification, this 
35 invention is not limited to the exact construction and 

operation shown and described herein. Accordingly, all 
suitable modifications and equivalents that may be 
resorted to in light of disclosure of this specification 
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are considered to fall within the scope of the invention 
as defined by the following claims. 
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WHAT IS CLAIMED IS: 

1 1. A system for determining the present time 

2 location and orientation of a moveable first object with 

3 respect to a moveable second object and for graphically 

4 indicating the corresponding position and orientation of 

5 said first object on a previously taken image of said 

6 second object, which comprises: 

7 a present time three-dimensional fixed coordinate 

8 system; 

9 at least three radiation sensor means in fixed 

10 spacial relationship to said present time three- 

11 dimensional fixed coordinate system which is spaced from 

12 said movable objects; 

13 said moveable first and second objects located within 

14 said fixed coordinate system; 

15 a three-dimensional local coordinate system, which 

16 remains fixed with respect to said second object, but is 

17 movable, along with said second object, within said fixed 

18 coordinate system; 

19 at least three non-coil inear radiation emitter means 

20 in fixed spacial relationship to said second object and at 

21 known coordinates in said local coordinate system; 

22 previously taken three-dimensional image data which 

23 geometrically describe said second object; 

24 at least two spaced apart radiation emitter means 

25 disposed on said first object; 

26 first movement means to move said first object 

27 relative to said second object and to said fixed 
2lBr"""''"'""^'coS ^" 

29 second movement means to move said second object 

30 relative to said first object and to said fixed coordinate 

31 system; 

32 means to transmit radiation between said radiation 

33 emitter means on said first and second objects and said 

34 radiation sensor means in fixed relation to said fixed 
3 5 coordination system; 
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36 means to distinguish between radiation emitted from 

37 any one radiation emitter means from radiation emitted 

38 from all of the other radiation emitter means; 

39 means to independently determine the location of each 

40 of said radiation emitter means on said first object as a 

41 function of said radiation being transferred between at 

42 least two of said radiation emitter means on said first 

43 object and said radiation sensor means in fixed relation 

44 to said fixed coordination system; 

45 means to determine the position and orientation of 

46 any point on said first object in relation to said fixed 

47 coordinate system by integrating the determined location 

48 of at least two of said radiation emitter means on said 

49 first object; 

50 means to independently determine the location of each 

51 of said radiation emitter means associated with said 

52 second object as a function of said radiation being 

53 transferred between said radiation emitter means 

54 associated with said second object and said radiation 

55 sensor mean in fixed relation to said fixed coordination 

56 system; 

57 means to determine the position and orientation of 

58 said second object in said fixed coordinate system by 

59 integrating the determined location of said radiation 

60 emitter means associated with said second object; 

61 means to determine the position and orientation of 

62 said second object in said local coordinate system; 

63 means to orient said previously taken three 

64 dimensional image data of said second object in said local 

65 coordination system such as to match the present time 
-6^^ orientation of «ai* second object wi^h the 

67 previously taken said image data; 

68 means to integrate the present time determined 

69 position and orientation of said second object with the 

70 present time determined position and orientation of said 

71 first object in the same fixed coordinate system whereby 

72 integrating the present time determined position and 

73 orientation of said first object in relation to said 
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74 previously taken three dimensional image data; 

75 means to, in present time, repeatedly determine the 

76 position and orientation of said first object with 

77 sufficient frequency to enable displaying the movement of 

78 said first object in relation to said second object; and 

79 means to, in present time, repeatedly determine the 

80 position and orientation of said second object with 

81 sufficient frequency to enable correlating a moved 

82 position and orientation of said second object with a view 

83 of said previously taken three dimensional image data 

84 which is consistent with the moved position and 

85 orientation of said second object and allows for the 

86 correct present time positioning of an image corresponding 

87 to the present time position and orientation of said first 

88 object in relation to said previously taken image data; 

89 wherein, as a consequence of said repeated 

90 determinations and integrations of the positions and 

91 orientations of said . first and second objects, and 

92 correlation of said position and orientation of said 

93 second object with said previously taken three dimensional 

94 image data, an image representative of said first obj ct 

95 is correctly positioned in present time on said previously 

96 taken image data regardless of the movement of said first 

97 and/or second objects in relation to said fixed coordinate 

98 system. 

1 2« A system as claimed in claim 1 wherein said 

2 first and second objects intersect, which system further 

3 comprises means to display so much of said previously 

4 taken three dimensional image data as corresponds to the 

,^ g ---£^^^^:^£j:^g ectsi ," 

6 and means to display said representation of said first 

7 object on said image display in its correct present time 

8 position. 

1 3 , A system as claimed in claim 1 wherein said 

2 frequency of determining the position and orientation of 

3 said second object is about 0.01 to 100 times, the 
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4 frequency of determining the position and orientation of 

5 said first object. 

1 4. A system as claimed in claim 1 wherein the 

2 position and orientation of said second object, the 

3 integration of the position and orientation of said second 

4 ob j ect and ^said image data , and the position and 

5 orientation of said first object are all determin d 

6 substantially continuously, 

1 5, The system as claimed in claim 1 wherein 

2 said previously taken three dimensional image data 

3 comprise a succession of substantially two dimensional 

4 slices at previously determined, known locations and 

5 orientations through said second object. 

1 6, The system as claimed in claim 1 wherein 

2 said radiation emitter means are disposed on said first 

3 and second objects and said sensor means are spaced from 

4 said first and second objects and are disposed in fixed 

5 relationship to said fixed coordinate system. 

1 7. The system as claimed in claim 1 wherein 

2 said sensor means are disposed on said first and second 

3 objects and said emitter means are spaced from said first 

4 and second objects and are disposed in fixed relationship 

5 to said fixed coordinate system. 

1 8. The system as claimed in claim 1 wherein 

2 said radiation is electromagnetic radiation. 

1 9. The system as claimed in claim 8 wherein 

2 said radiation is light. 

1 10. The system as claimed in claim 9 wherein 

2 light of different wave lengths is emitted from each 

3 different emitter means. 
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1 11. The system as claimed in claim 8 wherein 

2 said radiation is puis radiated from different of said 

3 emitter means sequentially in a repeating order. 

1 12. The system as claimed in claim 1 wherein 

2 said radiation is sound. 

1 13. The system as claimed in claim 1 wherein 

2 said radiation emitter means and said sensor means 

3 comprise means for detecting linear and rotational 

4 inert ial motion. 

1 14. The system as claimed in claim 1 wherein 

2 said previously taken image data are taken-^ by a means 

3 which is not suitable for use during a surgical procedure. 

1 15. The system as claimed in claim 1 wher in 

2 said first object is a surgical probe and said second 

3 object is a surgical patient, wherein said surgical probe 

4 is adapted to be partially inserted into said patient such 

5 that said inserted portion is not visible to an inserter 

6 thereof. 

1 16. The system as claimed in claim 15 wherein 

2 at least two emitter means are disposed on so much of said 

3 first object as is not inserted into said patient, and 

4 wherein said so disposed emitter means on said first 

5 object are a known distance from a tip of said probe which 

6 is adapted for insertion into said patient. 

. .. .... .-..^^^^^ clalBSd" in czXainTX whereiri 

2 said image data are selected from the group consisting of 

3 magnetic resonant image data, computed tomography data, 

4 ultra sound data, or X-radiation data. 

1 18. The syst m as claim d in claim 1 wherein 

2 said second object is a pati nt, and said first object is 

3 an operating microscope, and wherein said image data are 
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4 projected onto the field of said operating microscope. 

1 19. The system as claimed in claim 1 wherein 

2 said second object is a part of the anatomy of a human* 

1 20. The system as claimed in claim 19 wherein 

2 the part of the anatomy is a head. 

1 21. The system as claimed in claim 19 wherein 

2 there are reference points on said second object, which 

3 are distinguishing anatomical features / which are in fixed 

4 spacial relationship to said emitter means associated with 

5 said second object. 

1 22. The system as claimed in claim 1 operated 

2 intermittently, but sufficiently frequently to depict the 

3 relative movement of said first object relative to said 

4 second ob j ect , and the movement of said second ob j ect 

5 relative to said fixed coordinate system. 

1 23. The system as claimed in claim 2 including 

2 means for monitoring in present time the movement of said 

3 first object in true relation to said previously taken 

4 three dimensional image data. 

1 24. The system as claimed in claim 1 including 

2 means for selecting the slice of said previously taken 

3 image data which corresponds to the present time location 

4 of said first object such that a representation of said 

5 first object is correctly disposed on said image slice in 

1 25. The system as claimed in claim 1 including 

2 means to freeze a display of a previously taken image and 

3 a representation of said first object for a finite period 

4 of time. 



1' 



26. The system as claimed in claim 1 including 
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2 means to continuously display a series of slices of said 

3 previously taken three dimensional data which correspond 

4 to a changing position of said first object relative to 

5 said second object, and which correctly reflect movement 

6 of said second object relative to said fixed coordinate 

7 system in present time, 

1 27. A system for determining the present time 

2 location and orientation of a moveable first object with 

3 respect to a second object and for graphically indicating 

4 the corresponding position and orientation of said first 

5 object on a previously taken iiaage of said second object, 

6 which comprises: 

7 a present time three-dimensional fixed coordinate 

8 system; 

9 at least three radiation sensor means in fixed 

10 spacial relationship to said present time three- 

11 dimensional fixed coordinate system which are spaced from 

12 said objects; 

13 said first and second objects located within said 

14 fixed coordinate system; 

15 at least three non-collinear fiducial markers in 

16 fixed spacial relationship to said second object and at 

17 known coordinates in said local coordinate system; 

18 a three dimensional local coordinate system which is 

19 fixed in relation to said second object; 

20 previously taken three-dimensional image data which 

21 geometrically describe said second object including said 

22 fiducial markers; 

23 at least two spaced apart electromagnetic radiation 

25 first movement means to move said first object 

26 relative to said second object and to said fixed 

27 coordinate system; 

28 means to transmit radiation between said radiation 

29 emitter means on said first object and said radiation 

30 sensor means in fixed relation to said fixed coordination 

31 system; 
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32 means to distinguish radiation emitted from any one 

33 radiation emitter means from radiation emitted from all of 

34 the other radiation emitter means; 

35 means to independently determine the location of each 

36 of said radiation emitter means on said first object as a 

37 function of said radiation being transferred between at 

38 least two of said radiation emitter means on said first 

39 object and said radiation sensor means in fixed relation 

40 to said fixed coordination system; 

41 means to determine the position and orientation of 

42 any point on said first object in relation to said fixed 

43 coordinate system by integrating the determined location 

44 of at least two of said radiation emitter means on said 

45 first object; 

46 means to independently determine the location of each 

47 of said fiducial markers on said second object in said 

48 fixed coordinate system; 

49 means to determine the position and orientation of 

50 said second object in said fixed coordinate system by 

51 integrating the determined location of said fiducial 

52 markers on said second object; 

53 means to orient said previously taken three 

54 dimensional image data of said second object such as to 

55 match the present time position and orientation of said 

56 second object with the previously taken said image data; 

57 means to integrate the present time determined 

58 position and orientation of said second object with the 

59 present time determined position and orientation of said 

60 first object in the same fixed coordinate system whereby 

61 integrating the present time determined position and 

63 previously taken three dimensional image data; and 

^4 means to, in present time, repeatedly determine the 

65 positions and orientations of said first object with 

66 sufficient frequency to enable displaying the movement of 

67 said first object in relation to said second object; 

€8 wherein, as a consequence of said repeated 

69 determinations and integrations of the positions and 
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70 orientations of said first object, and correlation of said 

71 positions and orientations of said second object with said 

72 previously taken three dimensional image data, an image 

73 representative of said first object is correctly 

74 positioned in present time on said previously taken image 

75 data regardless of the movement of said first object in 

76 relation to said fixed coordinate system. 

1 28.. A system as claimed in claim 27 wherein 

2 said first and second objects intersect, which system 

3 further comprises means to- display so much of said 

4 previously taken three dimensional- imjage data as 

5 corresponds to the intersecting location of said first and 

6 second objects, and means to display said representation 

7 of said first object oh said image display in its correct 

8 present time position. 

1 29. A system as claimed in claim 27 wherein 

2 the integration of the position and orientation of said 

3 second object and said image data, the position and 

4 orientation of said first object^ and the correlation of 

5 the position and orientation of said first object with 

6 said image data are all determined sxibstantially 

7 continuously. 

1 30. The system as claimed in claim 27 wherein 

2 said previously taken three dimensional image data 

3 comprise a succession of substantially two dimensional 

4 slices through said second object. 

^v ... rr^~ . 2 ^:^'r. rpfj^jg: isyst§2V 3^ cral:med"^^ "wHereitf 

2 said radiation emitter means are disposed on said first 

3 object and said sensor means are spaced from said first 

4 and second objects and are disposed in fixed relationship 

5 to said fixed coordinate system. 

1 32. The system as claimed in claim 27 wherein 

2 said sensor means are disposed on said first object and 
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3 said radiation emitter means are spaced from said first 

4 object and are disposed in fixed relationship to said 

5 fixed coordinate system, 

1 33. The system as claimed in claim 27 wherein 

2 said radiation is light. 

1 34. The system as claimed in claim 33 wherein 

2 said radiation is visible light* 

1 35. The system as claimed in claim 33 wherein 

2 light of different wave lengths is emitted from each 

3 different emitter means. 

1 36. The system as claimed in claim 33 wherein 

2 said light is pulse radiated from different of said 

3 radiation emitter means sequentially in a repeating order. 

1 37. The system as claimed in claim 27 wherein 

2 said radiation emitter means and said sensor means 

3 comprise means for detecting linear and rotational 

4 inertial motion. 

1 38. The system as claimed in claim 27 wherein 

2 said previously taken image data are taken by a means 

3 which is not suitable for use during a surgical procedure. 

1 39. The system as claimed in claim 27 wherein 

2 said first object is a surgical probe and said second 

3 object is a surgical patient, wherein said surgical probe 

. . ^ - . -^.^^ . ^ ...rr adapted t c~ be " part ia^Il y ^ins^rte<r IhtCPS a id "patient ^ch ^ ' 

5 that said inserted portion is not visible to an inserter 

6 thereof. 

1 40. The system as claim d in claim 39 wh rein 

2 at least two emitter means are disposed on so much of said 

3 first object as is not inserted into said patient, and 

4 wherein said so disposed emitter means on said first 
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5 object are a known distance from a tip of said probe which 

6 is adapted for insertion into said patient. 

1 - 41. The system as claimed in claim 27 wherein 

2 said image data are selected from the group consisting of 

3 magnetic resonant image data, computed tomography data, 

4 ultra sound data, or X-radiation data. 

1 42. The system as claimed in claim 27 wherein 

2 said second object is a patient, and said first object is 

3 an operating microscope, and wherein said image data are 

4 projected onto the field of said operating microscope. 

1 43. The system as claimed in claim 27 wherein 

2 said second object is a part of the anatomy of a human. 

1 44 . The system as claimed in claim 43 wherein 

2 the part of the anatomy is a head. 

1 45. The system as claimed in claim 43 wherein 

2 there are reference points on said second object," which 

3 are distinguishing anatomical features, which are in fixed 

4 spacial relationship to said second object. 

1 46. The system as claimed in claim 27 operated 

2 intermittently, but sufficiently frequently to depict the 

3 relative movement of said first object relative to said 

4 second object. 

1 47. The system as claimed in claim 28 including 

2^ time the mdve^ of said 

3 first object in true relation to said previously taken 

4 three dimensional image data. 

1 48.. The system as claimed in claim 27 including 

2 means for selecting the slice of said previously taken 

3 image data which corresponds to the present time location 

4 of said first object such that a representation of said 
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5 first object is correctly disposed on said image slice in 

6 present time. 

1 49. The system as claimed in claim 27 including 

2 means to freeze a display of a previously taken image and 

3 a representation of said first object for a finite period 

4 of time. 

1 50. The system as claimed in claim 27 including 

2 means to continuously display a series of slices of said 

3 previously taken three dimensional data which correspond 

4 to a changing position of said first object relative to 

5 said second object> and which correctly reflect the 

6 position and orientation of said second object relative to 

7 said fixed coordinate system in present time. 

1 51. In a system for determining the present 

2 time location and orientation of a moveable first object 

3 with respect to a second object and for graphically 

4 indicating the corresponding position and orientation of 

5 said first object on a previously taken image of said 

6 second object, which comprises: 

7 a present time three-*dimensional fixed coordinate 

8 system; 

9 at least three radiation sensor means, in fixed 

10 spacial relationship to said present time three- 

11 dimensional fixed coordinate system, which are spaced from 

12 said objects; 

13 said first and second ob j ects located within said 

14 fixed coordinate system; 

. . ^--.r.^^^ . 25 - V three disiensionair lodar coordlTtat^ syst^ which" Is - ^ 

16 fixed in relation to said second object; 
- ^'7 at least three non-collinear fiducial markers in 

18 fixed spacial relationship to said second object and at 

19 known coordinates in said local coordinate system; 

20 previously taken three-dimensional image data which 

21 geometrically describe said second object including said 

22 fiducial markers in said fixed coordinate system; 
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23 at least two spaced apart radiation emitter means 

24 disposed on said first object; 

25 first movement means to move said first object 

26 relative to said second object; 

27 means to transmit radiation between said radiation 

28 emitter means on said first object and said radiation 

29 sensor means in fixed relation to said fixed coordinate 

30 system; 

31 means to distinguish radiation emitted from any one 

32 radiation emitter means from radiation emitted from all of 

33 the other radiation emitter means; 

34 means to independently determine the location of each 

35 of said radiation emitter means on said first object as a 

36 function of said radiation being transferred between at 

37 least two of said radiation emitter means on said first 

38 object and said radiation sensor means in fixed relation 

39 to said fixed coordinate system; 

40 means to determine the position and orientation of 

41 any point on said first object in relation to said fixed 

42 coordinate system by integrating the determined location 

43 of at least two of said radiation emitter means on said 

44 first object; 

45 means to independently determine the location of each 

46 of said fiducial markers on said second object; 

47 means to determine the position and orientation of 

48 said second object in said fixed coordinate system by 

49 integrating the determined location of said fiducial 

50 markers on said second object into said fixed coordinate 

51 system; 

52 means to orient said previously taken three 

54 match the present time position and orientation of said 

55 second object with the previously taken said image data; 

56 means to integrate the present time determined 

57 position and orientation of said second object with the 

58 present time determin d position and orientation of said 

59 first object in the same fixed coordinate system whereby 

60 integrating the present time determined position and 
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61 orienta'bion of said first object in relation to said 

62 previously taken three dimensional image data; and 

63 means to, in present time, repeatedly determine the 

64 position and orientation of said first object with 

65 sufficient frequency to enable displaying the movement of 

66 said first object in relation to said second object; 

67 wherein as a consequence of said repeated 

68 determinations and integrations of the positions and 

69 orientations of said first object, and correlation of said 

70 position and orientation of said second object with said 

71 previously taken three dimensional image data, an image 

72 representative of said first object is correctly 

73 positioned in present time on said previously taken image 

74 data regardless of the movement of said first object in 

75 relation to said fixed coordinate system; 

76 the improvement which comprises said radiation 

77 being electromagnetic radiation. 

1 52. In a system for determining the present 

2 time location and orientation of a moveable first object 

3 with respect to a moveable second object and for 

4 graphically indicating the corresponding position and 

5 orientation of said first object on a previously taken 

6 image of said second object, which comprises: 

7 a present time three-dimensional fixed coordinate 

8 system ; 

9 at least three radiation sensor means in fixed 

10 spacial relationship to said present time three- 

11 dimensional fixed coordinate system which is spaced from 

12 said movable objects; 

. . ...^..^ -^satd moveetble f xrst^ and^econd-ob^ects ^cate*^ithin - 

14 said fixed coordinate system; 

15 a three-dimensional local coordinate system, which 

16 remains fix d with respect to said second object, but is 

17 movable, along with said second object, within said fixed 

18 coordinate system; 

19 at least three non-collinear fiducial marker means in 

20 fixed spacial relationship to said second object and at 
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21 known coordinates in said local coordinate system; 

22 previously taken three-dimensional image data which 

23 geometrically describe said second object; 

24 at least two spaced apart radiation emitter means 

25 disposed on said first object; 

26 first movement means to move said first object 

27 relative to said second object and to said fixed 

28 coordinate system; 

29 second movement means to move said second object 

30 relative to said first object and to said fixed coordinat 

31 system; 

32 means to transmit radiation between said radiation 

33 emitter means on said first object and said radiation 

34 sensor means in fixed relation to said fixed coordination 

35 system; 

36 means to distinguis;h radiation emitted from any one 

37 radiation emitter means from radiation emitted from all of 

38 the other radiation emitter means; 

39 means to independently determine the location of each 
4 0 of said radiation emitter means on said first object as a 

41 function of said radiation being transferred between at 

42 least two of said radiation emitter means on said first 

43 object and said radiation sensor means in fixed relation 

44 to said fixed coordination system; 

45 means to determine the position and orientation of 
4 6 any point on said first object in relation to said fixed 

47 coordinate system by integrating the determined location 

48 of at least two of said radiation emitter means on said 

49 first object; and 

50 in present time, repeatedly determining the position 
. "31: " and""Sr^ 

52 frequency to enable displaying the movement of said first 

53 object in relation to said second object; 

54 the improvement which comprises: 

55 radiation emitter means associated in fixed, known 

56 spacial relationship to said fiducial markers; 

57 means to determine the location of said fiducial 

58 markers on said second object as a function of said 
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59 radialiion being transferred between said radiation emitter 

60 means associated with said second object and said 

61 radiation sensor means in fixed relation to said fixed 

62 coordinate system; 

63 means to determine the position and orientation of 

64 said second object in said fixed coordinate system by 

65 integrating ^the determined location of said radiation 

66 emitter means associated with said second object; 

67 means to determine the position and orientation of 

68 said second object in said local coordinate system; 

69 means to orient said previously taken three 

70 dimensional image data of said second object in said local 

71 coordination system such as to match the present time 

72 position and orientation of said second object with the 

73 previously taken said image data; 

74 means to integrate the present time determined 

75 position and orientation of said second object with the 

76 present time determined position and orientation of said 

77 first object in the same fixed coordinate system whereby 

78 integrating the present time determined position and 

79 orientation of said first object in relation to said 

80 previously taken three dimensional image data; and 

81 means to, in present time, repeatedly determine the 

82 position and orientation of said second object with 

83 sufficient frequency to enable correlating a moved 

84 position and orientation of said second object with a view 

85 of said previously taken three dimensional image data 

86 which is consistent with the moved position and 

87 orientation of said second object and allows for the 

88 correct present time positioning of an image corresponding 
-B9^^ - - to i:he present t:ime pas Itlon amd x>rientation-of ^a id f irs t 

90 object in relation to said previously taken image data. 

1 53. A method of determining the present time 

2 location and orientation of a moveable first object with 

3 respect to a moveable second object and for graphically 

4 indicating the corresponding position and orientation of 

5 said first object on a previously taken image of said 
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6 second object, which comprises: 

7 defining a present time three-dim nsional global 

8 fixed coordinate system; 

9 providing at least three radiation sensor means in 

10 fixed spacial relationship to said present time three- 

11 dimensional fixed coordinate system which is spaced from 

12 said movable objects; 

13 disposing said moveable first and second objects 

14 located within said fixed coordinate system; 

15 defining a three-dimensional local coordinate system, 

16 which remains fixed with respect to said second object, 

17 but is movable, along with said second object, within said 

18 fixed coordinate system; 

19 providing at least three non-collinear radiation 

20 emitter means in fixed spacial relationship to said second 

21 object and at known coordinates in said local coordinate 

22 system; 

23 taking, at a previous time, three-dimensional image 

24 data which geometrically describe said second object; 

25 providing at least two spaced apart radiation emitter 

26 means disposed on said first object; 

27 providing first movement means to move said first 

28 object relative to said second object and to said fixed 

29 coordinate system; 

30 providing second movement means to move said second 

31 object relative to said first object and to said fixed 

32 coordinate system; 

33 transmitting radiation between said radiation emitter 

34 means on said first and second objects and said radiation 

35 sensor means in fixed relation to said fixed coordination 

-y^-'^-^-^^Ysi^r^''^^'''''^^^^^ ..^ ■ 

37 distinguishing radiation emitted from any one 

38 radiation emitter means from radiation emitted from all of 

39 the other radiation emitter means; 

40 independently determining the location of each of 

41 said radiation emitter means on said first object as a 

42 function of said radiation being transferred between at 

43 least two of said radiation emitter means on said first 
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44 object and said radiation sensor means in fixed relation 

45 to said fixed coordination system; 

46 determining the position and orientation of any point 

47 on said first object in relation to said fixed coordinate 

48 system by integrating the determined location of at least 

49 two of said radiation emitter means on said first object; 

50 independently determining the location of each of 

51 said radiation emitter means associated with said second 

52 object as a function of said radiation being transferred 

53 between said radiation emitter means associated with said 

54 second object and said radiation sensor means in fixed 

55 relation to said fixed coordination system; 

56 determining the position and orientation of said 

57 second object in said fixed coordinate system by 

58 integrating the determined location of said radiation 

59 emitter means associated with said second object; 

60 determining the position and orientation of said 

61 second object in said local coordinate system; 

62 orienting said previously taken three dimensional 

63 image data of said second object in said local 

64 coordination system such as to match the present time 

65 position and orientation of said second object with the 

66 previously taken said image data; 

67 integrating the present time determined position and 

68 orientation of said second object with the present time 

69 determined position and orientation of said first object 

70 in the same fixed coordinate system whereby integrating 

71 the present time determined position and orientation of 

72 said first object in relation to said previously taken 

73 three dimensional image data; 
^^. .^^^ V ..... .. _74^ ^ ^. - - in present^ime^r 5^^E>^a^ 

75 and orientation of said first object with sufficient 

76 frequency to enable displaying the movement of said first 

77 object in relation to said second object; and 

78 in present time, repeatedly determining the position 

79 and orientation of said second object with sufficient 

80 frequency to nable correlating a moved position and 

81 orientation of said second object with a view of said 
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82 previously taken three dimensional image data which is 

83 consistent with the moved position and orientation of said 

84 second object and allows for the correct present time 

85 positioning of an image corresponding to the present time 

86 position and orientation of said first object in relation 

87 to said previously taken image data; 

88 wherein, as a consequence of said repeatedly 

89 determining and integrating the positions and orientations 

90 of said first and second objects, and correlating said 

91 position and orientation of said second object with said 

92 previously taken three dimensional image data, positioning 

93 an image representative first object correctly in present 

94 time on said previously taken image data regardless of the 

95 movement of said first and/ or second objects in relation 

96 to said fixed coordinate system* 

1 54. A method as claimed in claim 53 wherein 

2 said first and second objects intersect, which method 

3 further comprises displaying so much of said previously 

4 taken three dimensional image data as corresponds to the 

5 intersecting location of said first and second objects, 

6 and displaying said representation of said first object on 

7 said image display in its correct present time position. 

1 55* A method as claimed in claim 53 wherein 

2 including determining the position and orientation of said 

3 second object at a frequency which is about 0.01 to 100 

4 times the frequency of determining the position and 

5 orientation of said first object. 

2 substantially continuouisly determining the position and 

3 orientation of said second object, the integration of the 

4 position and orientation of said second object and said 

5 image data, and the position and orientation of said first 

6 object. 

1 57. The method as claimed in claim 53 including 
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2 taking three dimensional image data which comprise a 

3 succession of substantially two dimensional slices at a 

4 previous time at known locations and orientations through 

5 said second object. 

1 58. The method as claimed in claim 53 including 

2 disposing said radiation emitter means on said first and 

3 second objects, spacing said sensor means a distance from 

4 said first and second objects, and disposing said sensor 

5 means in fixed relationship to said fixed coordinate 

6 system. 

1 59. The method as claimed in claim 53 including 

2 disposing said sensor means on said first and second 

3 objects, spacing said emitter means a distance from said 

4 first and second objects, and disposing said radiation 

5 emitters in fixed relationship to said fixed coordinate 

6 system. 

1 60. The method as claimed in claim 53 wherein 

2 said radiation is electromagnetic radiation 

1 61. The method as claimed in claim 60 wherein 

2 said radiation is light. 

1 62. The method as claimed in claim 61 including 

2 emitting light of different wave lengths from each 

3 different emitter means. 

1 63. The method as claimed in claim 60 including 

3 emitter means sequentially in a repeating order. 

1 64. The method as claimed in claim 53 wherein 

2 said radiation is sound. 

1 65. The method as claimed in claim 53 including 

2 detecting linear and rotational inert ial motion through 
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3 said radiation eroitt:er laeans and said sensor means. 

1 66* The method as claimed in claim 53 including 

2 taking said previously taken image data by a means which 

3 is not suitable for use during a surgical procedure. 

1 67. The method as claimed in claim 53 wherein 

2 said first object is a surgical probe and said second 

3 object^ is a surgical patient, and partially inserting said 

4 surgical probe within said patient such that said inserted 

5 portion is not visible to an inserter thereof, 

1 68. The method as claimed in claim 67 including 

2 disposing at least two emitter means on so much of said 

3 first object as is not inserted into said patient, and 

4 disposing said exposed portion of said emitter means on 

5 said first object a known distance from a tip of said 

6 probe which is adapted for insertion into said patient. 

1 69. The method as claimed in claim 53 including 

2 selecting said image data from the group consisting of 

3 magnetic resonant image data, computed tomography data, 

4 ultra sound data, or X-radiation data. 

1 70. The method as claimed in claim 53 wherein 

2 said second object is a patient, and said first object is 

3 an operating microscope, and including projecting said 

4 image data onto the field of an operating microscope. 

1 71. The method as claimed in claim 53 wherein 

' 2 " "^"^ o^' a liumanV 

1 72. The method as claimed in claim 71 wherein 

2 the part of the anatomy is a head. 

1 73. The system as claimed in claim 71 wherein 

2 there ar fiducial reference points on said second object, 

3 which are distinguishing anatomical features, which are in 
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4 fixed ^ spacial relat:ionship to said emitter means 

5 associated with said second object. 

1 74. The method as claimed in claim 53 which is 

2 operated intermittently, but sufficiently frequently to 

3 depict the relative movement of said first object relative 

4 to said second object, and the movement of said sec nd 

5 object relative to said fixed coordinate system. 

1 75. The method as claimed in claim 54 including 

2 monitoring, in present time, the movement of said first 

3 object in true relation to said previously taken three 

4 dimensional image data. 

1 76. The method as claimed in claim 53 including 

2 selecting the slice of said previously taken image data 

3 which corresponds to the present time location of said 

4 first object, and correctly positioning a representation 

5 of said first object on said selected image slice in 

6 present time. 

1 77. The method as claimed in claim 53 including 

2 freezing a display of a previously taken image and a 

3 representation of said first object thereon for a finite 

4 period of time. 

1 78. The method as claimed in claim 53 including 

2 continuously displaying a series of slices of said 

3 previously taken three dimensional data which correspond 

4 to a changing position of said first object relative to 

^^-^^^^ - ^ ^^^^^^ . 

6 movement of said second object relative to said fixed 

7 coordinate system in present time. 
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